Found 50 jobs
Ignoring malparsed backtrace:  ceph version 16.0.0-8280-g81d0278a (81d0278a02295c5a0d481303def7f69477d4ff55) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe1) [0x55e7f4a84179]
,  2: (KernelDevice::_aio_thread()+0xbb9) [0x55e7f553fa19]

Ignoring malparsed backtrace: 2020-12-16T23:46:14.841+0000 7f5c623c9700  1 -- 172.21.15.71:0/2950736834 >> v2:172.21.15.93:3300/0 conn(0x7f5c5c150930 msgr2=0x7f5c5c150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down
, 2020-12-16T23:46:14.841+0000 7f5c623c9700  1 --2- 172.21.15.71:0/2950736834 >> v2:172.21.15.93:3300/0 conn(0x7f5c5c150930 0x7f5c5c150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:46:14.841+0000 7f5c623c9700  1 -- 172.21.15.71:0/2950736834 --> v2:172.21.15.71:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c0bcf20 con 0x7f5c5c0bbde0

Ignoring malparsed backtrace: 2020-12-16T23:46:14.913+0000 7f5c728c6740  1 -- 172.21.15.71:0/1049512616 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1088980 con 0xe1bbb0
, 2020-12-16T23:46:14.913+0000 7f5c387f8700  1 -- 172.21.15.71:0/1049512616 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c54010380 con 0xe1bbb0
, 2020-12-16T23:46:14.917+0000 7f5c728c6740 10 client.5858 fetch_fsmap learned FSMap version 8

Ignoring malparsed backtrace: 2020-12-16T23:46:15.045+0000 7f5c728c6740  1 client.5976 shutdown
, 2020-12-16T23:46:15.045+0000 7f5c728c6740  1 -- 172.21.15.71:0/3334697049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x10b14a0 msgr2=0x10d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:46:15.045+0000 7f5c728c6740  1 --2- 172.21.15.71:0/3334697049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x10b14a0 0x10d5dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f5c540104b0 tx=0x7f5c5402d000).stop

Ignoring malparsed backtrace: 2020-12-16T23:46:17.261 INFO:tasks.fwd_scrub.fs.[cephfs]:joining all scrubbers
, 2020-12-16T23:46:17.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json
, 2020-12-16T23:46:17.585+0000 7f3649cd5700  1  Processor -- start

Ignoring malparsed backtrace: 2020-12-16T23:46:18.757+0000 7f7c753ea700  1 -- 172.21.15.71:0/3798397484 wait complete.
, 2020-12-16T23:46:18.761+0000 7f7c753ea700  1  Processor -- start
, 2020-12-16T23:46:18.761+0000 7f7c753ea700  1 --  start start

Ignoring malparsed backtrace: 2020-12-16T23:46:18.817+0000 7f7c850e6740  1 --2- 172.21.15.71:0/474225848 >> [v2:172.21.15.93:6832/2009652031,v1:172.21.15.93:6833/2009652031] conn(0x19be060 0x19f60b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:46:18.817+0000 7f7c850e6740  1 --2- 172.21.15.71:0/474225848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x19b48a0 0x19bdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop
, 2020-12-16T23:46:18.817+0000 7f7c850e6740  1 --2- 172.21.15.71:0/474225848 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x172ac50 0x1a08350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:46:18.933+0000 7f7c753ea700  1 -- 172.21.15.71:0/834391321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c700619e0 msgr2=0x7f7c70160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:46:18.933+0000 7f7c753ea700  1 --2- 172.21.15.71:0/834391321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c700619e0 0x7f7c70160840 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f7c6000a840 tx=0x7f7c6001b000).stop
, 2020-12-16T23:46:18.937+0000 7f7c753ea700  1 -- 172.21.15.71:0/834391321 shutdown_connections

Ignoring malparsed backtrace: 2020-12-16T23:46:20.797+0000 7f905cb8b700  1 --2- 172.21.15.71:0/799456755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9058065030 0x7f90580bef90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f9050009c90 tx=0x7f90500091d0).stop
, 2020-12-16T23:46:20.797+0000 7f905cb8b700  1 -- 172.21.15.71:0/799456755 shutdown_connections
, 2020-12-16T23:46:20.797+0000 7f905cb8b700  1 --2- 172.21.15.71:0/799456755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9058065030 0x7f90580bef90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:46:20.881+0000 7f906c887740  1 -- 172.21.15.71:0/4284417446 --> [v2:172.21.15.93:6832/2009652031,v1:172.21.15.93:6833/2009652031] -- command(tid 0: { " p r e f i x " :   " g e t _ c o m m a n d _ d e s c r i p t i o n s " }) v1 -- 0x1c7f620 con 0x1eb8a30
, 2020-12-16T23:46:20.885+0000 7f9056ffd700  1 --2- 172.21.15.71:0/4284417446 >> [v2:172.21.15.93:6832/2009652031,v1:172.21.15.93:6833/2009652031] conn(0x1eb8a30 0x1eb9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0
, 2020-12-16T23:46:20.885+0000 7f9056ffd700  1 --2- 172.21.15.71:0/4284417446 >> [v2:172.21.15.93:6832/2009652031,v1:172.21.15.93:6833/2009652031] conn(0x1eb8a30 0x1eb9bc0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=181ee801c9a870f9 server_cookie=318ac2b7d3c1274e in_seq=0 out_seq=0

Ignoring malparsed backtrace: 2020-12-16T23:46:21.093+0000 7f906c887740  1 -- 172.21.15.71:0/2126434507 shutdown_connections
, 2020-12-16T23:46:21.093+0000 7f906c887740  1 -- 172.21.15.71:0/2126434507 wait complete.
, 2020-12-16T23:46:21.101+0000 7f906c887740 20 client.5923 trim_cache size 0 max 16384

Ignoring malparsed backtrace: 2020-12-16T23:46:24.233+0000 7f0ff90c4700  1 -- 172.21.15.71:0/3133674624 --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff41528a0 con 0x7f0ff409eeb0
, 2020-12-16T23:46:24.233+0000 7f0ff90c4700  1 -- 172.21.15.71:0/3133674624 --> v2:172.21.15.71:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40c2760 con 0x7f0ff409f2f0
, 2020-12-16T23:46:24.233+0000 7f0ff90c4700  1 -- 172.21.15.71:0/3133674624 --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40c2a00 con 0x7f0ff409fde0

Ignoring malparsed backtrace: 2020-12-16T23:46:25.720+0000 7f0ffb7b0700  1 --  start start
, 2020-12-16T23:46:25.720+0000 7f0ffb7b0700  1 --2-  >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ff40bd090 0x7f0ff4172450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:46:25.720+0000 7f0ffb7b0700  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ff415b440 0x7f0ff415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect

Ignoring malparsed backtrace: 2020-12-16T23:46:25.760+0000 7f0fd37fe700  1 -- 172.21.15.71:0/3910891063 <== mds.0 v2:172.21.15.93:6832/2009652031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2ae36c0 con 0x2d3c950
, 2020-12-16T23:46:25.760+0000 7f0fd37fe700 10 client.5941 handle_command_reply: tid=0
, 2020-12-16T23:46:25.760+0000 7f100a24a740  1 client.5941 shutdown

Ignoring malparsed backtrace: 2020-12-16T23:46:26.020+0000 7f0ffb7b0700  1 -- 172.21.15.71:0/2647303562 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f0fe007edd0 msgr2=0x7f0fe0081290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:46:26.020+0000 7f0ffb7b0700  1 --2- 172.21.15.71:0/2647303562 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f0fe007edd0 0x7f0fe0081290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f0ff000b190 tx=0x7f0ff0016000).stop
, 2020-12-16T23:46:26.020+0000 7f0ffb7b0700  1 -- 172.21.15.71:0/2647303562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ff415b440 msgr2=0x7f0ff415e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:46:27.748+0000 7fa32a59c700  1 -- 172.21.15.71:0/3705473282 >> v2:172.21.15.181:3300/0 conn(0x7fa32c067f00 msgr2=0x7fa32c151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down
, 2020-12-16T23:46:27.748+0000 7fa32a59c700  1 --2- 172.21.15.71:0/3705473282 >> v2:172.21.15.181:3300/0 conn(0x7fa32c067f00 0x7fa32c151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:46:27.748+0000 7fa32a59c700  1 -- 172.21.15.71:0/3705473282 >> v1:172.21.15.93:6789/0 conn(0x7fa32c1516f0 legacy=0x7fa32c064660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:46:27.820+0000 7fa30cff9700  1 -- 172.21.15.71:0/4211618615 <== mon.1 v2:172.21.15.93:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f5b620 con 0x1eccbb0
, 2020-12-16T23:46:27.820+0000 7fa30cff9700  1 -- 172.21.15.71:0/4211618615 <== mon.1 v2:172.21.15.93:3300/0 5 ==== fsmap(e 8) v1 ==== 1449+0+0 (secure 0 0 0) 0x7fa324078160 con 0x1eccbb0
, 2020-12-16T23:46:27.820+0000 7fa33f574740 10 client.5962 fetch_fsmap finished waiting for FSMap version 8

Ignoring malparsed backtrace: 2020-12-16T23:46:27.912+0000 7fa33f574740  1 client.5960 shutdown
, 2020-12-16T23:46:27.928+0000 7fa33f574740  1 -- 172.21.15.71:0/3352848670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x215f010 msgr2=0x214dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:46:27.928+0000 7fa33f574740  1 --2- 172.21.15.71:0/3352848670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x215f010 0x214dee0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fa3200075d0 tx=0x7fa320029000).stop

Ignoring malparsed backtrace: 2020-12-16T23:46:59.120+0000 7f6347fff700  1 -- 172.21.15.71:0/3780156945 shutdown_connections
, 2020-12-16T23:46:59.120+0000 7f6347fff700  1 --2- 172.21.15.71:0/3780156945 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f634807ec40 0x7f6348081100 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:46:59.120+0000 7f6347fff700  1 --2- 172.21.15.71:0/3780156945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6368150930 0x7f636814ef10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:46:59.712+0000 7f2602ca9740  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24af800 0x24b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:46:59.712+0000 7f2602ca9740  1 --  --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x23e00b0 con 0x22937b0
, 2020-12-16T23:46:59.712+0000 7f25f17aa700  1 --2-  >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x22937b0 0x25033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0

Ignoring malparsed backtrace: 2020-12-16T23:46:59.816+0000 7f25c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0
, 2020-12-16T23:46:59.816+0000 7f25c7fff700  1 -- 172.21.15.71:0/607220059 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25dc001190 con 0x22937b0
, 2020-12-16T23:46:59.816+0000 7f2602ca9740  1 -- 172.21.15.71:0/607220059 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25e4001cf0 con 0x22937b0

Ignoring malparsed backtrace: 2020-12-16T23:47:01.448+0000 7f9aa57fa700  1 -- 172.21.15.71:0/2571035454 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 100169+0+0 (secure 0 0 0) 0x7f9ab001c030 con 0x7f9ab8150930
, 2020-12-16T23:47:01.448+0000 7f9aa57fa700  1 --2- 172.21.15.71:0/2571035454 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f9aa007ef40 0x7f9aa0081400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:47:01.448+0000 7f9aa57fa700  1 -- 172.21.15.71:0/2571035454 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 7913+0+0 (secure 0 0 0) 0x7f9ab00a2460 con 0x7f9ab8150930

Ignoring malparsed backtrace: 2020-12-16T23:47:03.387 INFO:teuthology.orchestra.run.smithi071.stdout:
, 2020-12-16T23:47:03.388 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.71:6833/1991978088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1991978088},{"type":"v1","addr":"172.21.15.71:6833","nonce":1991978088}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":5},{"gid":4301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1079340338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1079340338},{"type":"v1","addr":"172.21.15.181:6835","nonce":1079340338}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-12-16T23:36:19.384335+0000","modified":"2020-12-16T23:36:22.473639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.93:6833/2009652031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2009652031},{"type":"v1","addr":"172.21.15.93:6833","nonce":2009652031}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]}
, 2020-12-16T23:47:03.392+0000 7fb3f18a2700  1 -- 172.21.15.71:0/1489797124 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7fb3cc07ecc0 msgr2=0x7fb3cc081180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:47:04.164+0000 7f248c817700  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ce5c50 0x1fc4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.71:48054/0 (socket says 172.21.15.71:48054)
, 2020-12-16T23:47:04.164+0000 7f248c817700  1 -- 172.21.15.71:0/629056006 learned_addr learned my addr 172.21.15.71:0/629056006 (peer_addr_for_me v2:172.21.15.71:0/0)
, 2020-12-16T23:47:04.164+0000 7f249d515740  1 --2- 172.21.15.71:0/629056006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x1f6f8a0 0x1f73190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect

Ignoring malparsed backtrace: 2020-12-16T23:47:04.276+0000 7f248d018700  1 --2- 172.21.15.71:0/3878576643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x1f6f8a0 0x1f66f80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f247800deb0 tx=0x7f2478022000).ready entity=mon.1 client_cookie=8f01fd42db2edc9a server_cookie=0 in_seq=0 out_seq=0
, 2020-12-16T23:47:04.276+0000 7f249d515740  1 -- 172.21.15.71:0/3878576643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f247800fa00 con 0x1f6f8a0
, 2020-12-16T23:47:04.300+0000 7f2466ffd700 10 client.6016 ms_handle_connect on v2:172.21.15.93:3300/0

Ignoring malparsed backtrace: 2020-12-16T23:47:05.060+0000 7f5f677fe700  1 -- 172.21.15.71:0/219147897 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 7913+0+0 (secure 0 0 0) 0x7f5f580a22a0 con 0x7f5f700618f0
, 2020-12-16T23:47:05.060+0000 7f5f6f7fe700  1 --2- 172.21.15.71:0/219147897 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f5f5407ed40 0x7f5f54081200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0
, 2020-12-16T23:47:05.064+0000 7f5f6f7fe700  1 --2- 172.21.15.71:0/219147897 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f5f5407ed40 0x7f5f54081200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f5f6800b460 tx=0x7f5f68006900).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0

Ignoring malparsed backtrace: 2020-12-16T23:47:06.288+0000 7faedf290740  1 --2- 172.21.15.71:0/2203841577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x249c800 0x249fcb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:06.288+0000 7faedf290740  1 --2- 172.21.15.71:0/2203841577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x22807b0 0x24a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop
, 2020-12-16T23:47:06.288+0000 7faedf290740  1 --2- 172.21.15.71:0/2203841577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x2212bb0 0x2499b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:47:06.456+0000 7faedf290740  1 --2- 172.21.15.71:0/1147436041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x24ee5e0 0x24a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:47:06.456+0000 7faedf290740  1 -- 172.21.15.71:0/1147436041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x24df5e0 con 0x22807b0
, 2020-12-16T23:47:06.456+0000 7faedf290740  1 -- 172.21.15.71:0/1147436041 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x2545740 con 0x2212bb0

Ignoring malparsed backtrace: 2020-12-16T23:47:07.836+0000 7f051d945700  1 --2- 172.21.15.71:0/2153076037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0518150930 0x7f0518148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0
, 2020-12-16T23:47:07.836+0000 7f051f3a8700  1 -- 172.21.15.71:0/2153076037 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05181492f0 con 0x7f0518150930
, 2020-12-16T23:47:07.836+0000 7f051f3a8700  1 -- 172.21.15.71:0/2153076037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0518149570 con 0x7f05180bbde0

Ignoring malparsed backtrace: 2020-12-16T23:47:10.048+0000 7f5fa8778700  1 -- 172.21.15.71:0/959216263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f7c003fd0 con 0x7f5f9804b780
, 2020-12-16T23:47:10.048+0000 7f5f967fc700  1 -- 172.21.15.71:0/959216263 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2486 (secure 0 0 0) 0x7f5fa00d8610 con 0x7f5f9804b780
, 2020-12-16T23:47:10.052 INFO:teuthology.orchestra.run.smithi071.stdout:

Ignoring malparsed backtrace: 2020-12-16T23:47:10.444+0000 7f890b7fe700  1 --2- 172.21.15.71:0/1531179472 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f88ec07ed10 0x7f88ec0811d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x1352160 tx=0x7f8900008040).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0
, 2020-12-16T23:47:10.444+0000 7f89208ad740  1 -- 172.21.15.71:0/3348923845 wait complete.
, 2020-12-16T23:47:10.448+0000 7f89208ad740  1 --  start start

Ignoring malparsed backtrace: 2020-12-16T23:47:10.560+0000 7f890affd700  1 --2- 172.21.15.71:0/167089949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x11a2c50 0x1472600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:10.560+0000 7f890affd700  1 -- 172.21.15.71:0/167089949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x142c8a0 msgr2=0x146c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down
, 2020-12-16T23:47:10.560+0000 7f890affd700  1 --2- 172.21.15.71:0/167089949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x142c8a0 0x146c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:47:10.964+0000 7f14667fc700  1 -- 172.21.15.71:0/43436407 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14740057f0 con 0x7f1478067bb0
, 2020-12-16T23:47:10.964+0000 7f14667fc700  1 -- 172.21.15.71:0/43436407 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 100169+0+0 (secure 0 0 0) 0x7f147401a060 con 0x7f1478067bb0
, 2020-12-16T23:47:10.968+0000 7f14667fc700  1 --2- 172.21.15.71:0/43436407 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f146007ee60 0x7f1460081320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect

Ignoring malparsed backtrace: 2020-12-16T23:47:11.696+0000 7fe94bfff700  1 -- 172.21.15.71:0/3024872501 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c012b20 con 0x19b67b0
, 2020-12-16T23:47:11.696+0000 7fe94bfff700  1 -- 172.21.15.71:0/3024872501 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c019040 con 0x19b67b0
, 2020-12-16T23:47:11.696+0000 7fe94bfff700  1 -- 172.21.15.71:0/3024872501 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c012e90 con 0x19b67b0

Ignoring malparsed backtrace: 2020-12-16T23:47:11.748+0000 7fe96affd700  1 --2-  >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x1948bb0 0x1c14510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53160/0 (socket says 172.21.15.71:53160)
, 2020-12-16T23:47:11.748+0000 7fe96affd700  1 -- 172.21.15.71:0/4094001288 learned_addr learned my addr 172.21.15.71:0/4094001288 (peer_addr_for_me v2:172.21.15.71:0/0)
, 2020-12-16T23:47:11.748+0000 7fe97fd93740  1 -- 172.21.15.71:0/4094001288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x1c2dee0 con 0x1948bb0

Ignoring malparsed backtrace: 2020-12-16T23:47:13.112+0000 7fc2277fe700  1 -- 172.21.15.71:0/3680065811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc228152e60 msgr2=0x7fc228165800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:47:13.112+0000 7fc2277fe700  1 --2- 172.21.15.71:0/3680065811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc228152e60 0x7fc228165800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:13.112+0000 7fc2277fe700  1 -- 172.21.15.71:0/3680065811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2280bfe10 msgr2=0x7fc2281696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:47:13.768+0000 7f6120d4c700  1 --2- 172.21.15.71:0/1840819838 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f60fc07ec40 0x7f60fc081100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f610c006ee0 tx=0x7f610c016000).stop
, 2020-12-16T23:47:13.768+0000 7f6120d4c700  1 -- 172.21.15.71:0/1840819838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f611c14e1a0 msgr2=0x7f611c157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:47:13.768+0000 7f6120d4c700  1 --2- 172.21.15.71:0/1840819838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f611c14e1a0 0x7f611c157260 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f611000c6e0 tx=0x7f6110007ae0).stop

Ignoring malparsed backtrace: 2020-12-16T23:47:14.112+0000 7f004bc1f740  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1627c50 0x18f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:47:14.112+0000 7f004bc1f740  1 --2-  >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x1695850 0x18f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:47:14.112+0000 7f004bc1f740  1 --2-  >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x18f0710 0x18f1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect

Ignoring malparsed backtrace: 2020-12-16T23:47:14.260+0000 7f0036d9d700  1 --2- 172.21.15.71:0/706445500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x1695850 0x194f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:14.260+0000 7f0036d9d700  1 -- 172.21.15.71:0/706445500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1607070 con 0x1627c50
, 2020-12-16T23:47:14.260+0000 7f0036d9d700  1 --2- 172.21.15.71:0/706445500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1627c50 0x18f00b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f0028079180 tx=0x7f002807b040).ready entity=mon.2 client_cookie=ff5019c9ca5daa6a server_cookie=0 in_seq=0 out_seq=0

Ignoring malparsed backtrace: 2020-12-16T23:47:15.244+0000 7f7d91ffb700  1 -- 172.21.15.71:0/3663770402 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 100169+0+0 (secure 0 0 0) 0x7f7da0011d00 con 0x7f7da4067bb0
, 2020-12-16T23:47:15.252+0000 7f7daa913700  1 -- 172.21.15.71:0/3663770402 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da405d820 con 0x7f7da4067bb0
, 2020-12-16T23:47:15.256+0000 7f7d91ffb700  1 --2- 172.21.15.71:0/3663770402 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f7d8407ec90 0x7f7d84081150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect

Ignoring malparsed backtrace: 2020-12-16T23:47:16.512+0000 7fc815c06740  1 --2- 172.21.15.71:0/3759993579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xfcb800 0xfcecb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:16.512+0000 7fc815c06740  1 --2- 172.21.15.71:0/3759993579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0xdaf7b0 0xfcfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:16.512+0000 7fc815c06740  1 --2- 172.21.15.71:0/3759993579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0xd41bb0 0xfc8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:47:16.680+0000 7fc7fffff700  1 -- 172.21.15.71:0/306018240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd41bb0 msgr2=0xfcf070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down
, 2020-12-16T23:47:16.680+0000 7fc7fffff700  1 --2- 172.21.15.71:0/306018240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0xd41bb0 0xfcf070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop
, 2020-12-16T23:47:16.680+0000 7fc7fffff700  1 -- 172.21.15.71:0/306018240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0xfc3570 msgr2=0x1070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:47:18.164+0000 7fa12e414700  1 -- 172.21.15.71:0/3002722709 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12815b840 con 0x7fa1280bae10
, 2020-12-16T23:47:18.164+0000 7fa11d7fa700  1 -- 172.21.15.71:0/3002722709 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120013910 con 0x7fa1280bae10
, 2020-12-16T23:47:18.168+0000 7fa12e414700  1 -- 172.21.15.71:0/3002722709 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa100004f60 con 0x7fa1280bae10

Ignoring malparsed backtrace: 2020-12-16T23:47:19.828+0000 7f927b59e700  1 --2- 172.21.15.71:0/4277007937 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f926007ee10 0x7f92600812d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:19.828+0000 7f927b59e700  1 --2- 172.21.15.71:0/4277007937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f927c151c90 0x7f927c0c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop
, 2020-12-16T23:47:19.828+0000 7f927b59e700  1 --2- 172.21.15.71:0/4277007937 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f927c0bfed0 0x7f927c14f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:47:20.196+0000 7fdc1e945740  1 --2-  >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x23b9c50 0x2693610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:47:20.196+0000 7fdc1e945740  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x2427850 0x26a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:47:20.196+0000 7fdc1e945740  1 --2-  >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x26438a0 0x26a3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect

Ignoring malparsed backtrace: 2020-12-16T23:47:20.328+0000 7fdc1e945740  1 -- 172.21.15.71:0/18156399 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc00000fd0 con 0x23b9c50
, 2020-12-16T23:47:20.328+0000 7fdbe3fff700  1 -- 172.21.15.71:0/18156399 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(29..29 src has 1..29) v4 ==== 7913+0+0 (secure 0 0 0) 0x7fdbfc015660 con 0x23b9c50
, 2020-12-16T23:47:20.328+0000 7fdbe3fff700  1 -- 172.21.15.71:0/18156399 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbfc015820 con 0x23b9c50

Ignoring malparsed backtrace: 2020-12-16T23:47:21.012+0000 7f1ac2ffd700  1 -- 172.21.15.71:0/2849512662 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad40199f0 con 0x7f1adc15b0d0
, 2020-12-16T23:47:21.012+0000 7f1ac2ffd700  1 -- 172.21.15.71:0/2849512662 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 100169+0+0 (secure 0 0 0) 0x7f1ad4029030 con 0x7f1adc15b0d0
, 2020-12-16T23:47:21.016+0000 7f1ae1903700  1 -- 172.21.15.71:0/2849512662 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab4004f60 con 0x7f1adc15b0d0

Ignoring malparsed backtrace: 2020-12-16T23:47:22.312+0000 7fce5aa3c740  1 -- 172.21.15.71:0/799192142 shutdown_connections
, 2020-12-16T23:47:22.312+0000 7fce5aa3c740  1 --2- 172.21.15.71:0/799192142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x16c5800 0x16c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop
, 2020-12-16T23:47:22.312+0000 7fce5aa3c740  1 --2- 172.21.15.71:0/799192142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x14a97b0 0x16c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:47:22.380+0000 7fce5aa3c740 10 client.0 ll_register_callbacks cb 0x171c000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0
, 2020-12-16T23:47:22.380+0000 7fce49d3e700  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x143bbb0 0x16c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.71:48470/0 (socket says 172.21.15.71:48470)
, 2020-12-16T23:47:22.380+0000 7fce49d3e700  1 -- 172.21.15.71:0/590280077 learned_addr learned my addr 172.21.15.71:0/590280077 (peer_addr_for_me v2:172.21.15.71:0/0)

Ignoring malparsed backtrace: 2020-12-16T23:47:24.004+0000 7fc5360f6700  1 --2- 172.21.15.71:0/3832168164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc51c004e40 0x7fc51c00b720 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:24.004+0000 7fc5360f6700  1 -- 172.21.15.71:0/3832168164 >> 172.21.15.71:0/3832168164 conn(0x7fc530062890 msgr2=0x7fc5300bdb60 unknown :-1 s=STATE_NONE l=0).mark_down
, 2020-12-16T23:47:24.008+0000 7fc5360f6700  1 -- 172.21.15.71:0/3832168164 shutdown_connections

Ignoring malparsed backtrace: 2020-12-16T23:47:25.004+0000 7f0df67fc700  1 -- 172.21.15.71:0/4108779784 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dfc01c580 con 0x7f0e08080520
, 2020-12-16T23:47:25.004+0000 7f0df67fc700  1 -- 172.21.15.71:0/4108779784 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc01c920 con 0x7f0e08080520
, 2020-12-16T23:47:25.004+0000 7f0e0759e700  1 -- 172.21.15.71:0/4108779784 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e08164d40 con 0x7f0e08080520

Ignoring malparsed backtrace: 2020-12-16T23:47:26.284+0000 7ff64f12b740  1 -- 172.21.15.71:0/3888544160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x1f3b8a0 msgr2=0x1f3ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:47:26.284+0000 7ff64f12b740  1 --2- 172.21.15.71:0/3888544160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x1f3b8a0 0x1f3ed50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7ff638151790 tx=0x7ff634007b60).stop
, 2020-12-16T23:47:26.288+0000 7ff6267fc700  1 -- 172.21.15.71:0/2747296826 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62800d8f0 con 0x7ff638060390

Ignoring malparsed backtrace: 2020-12-16T23:47:26.512+0000 7ff64f12b740  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1d1f850 0x1f84780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:47:26.512+0000 7ff64f12b740  1 --2-  >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x1f3b8a0 0x1f84c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:47:26.512+0000 7ff64f12b740  1 --  --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f39f80 con 0x1f3b8a0

Ignoring malparsed backtrace: 2020-12-16T23:47:27.116+0000 7f6b5659c700  1 --2- 172.21.15.71:0/675091979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b3c003ed0 0x7f6b3c00b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:27.116+0000 7f6b5659c700  1 -- 172.21.15.71:0/675091979 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c003230 con 0x7f6b3c004a80
, 2020-12-16T23:47:27.116+0000 7f6b56d9d700  1 --2- 172.21.15.71:0/675091979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b3c003ed0 0x7f6b3c00b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed!

Ignoring malparsed backtrace: 2020-12-16T23:47:28.392+0000 7f3f43855700  1 --  --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c166040 con 0x7f3f3c150930
, 2020-12-16T23:47:28.392+0000 7f3f43855700  1 --  --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c1662e0 con 0x7f3f3c060390
, 2020-12-16T23:47:28.392+0000 7f3f415f1700  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f3c060390 0x7f3f3c143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0

Ignoring malparsed backtrace: 2020-12-16T23:47:28.524+0000 7f3f522ef740  1 --2- 172.21.15.71:0/1415225839 >> [v2:172.21.15.93:6832/2009652031,v1:172.21.15.93:6833/2009652031] conn(0x1cfc9d0 0x1d4c8b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:28.524+0000 7f3f522ef740  1 --2- 172.21.15.71:0/1415225839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x1ad97b0 0x1d36d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:47:28.524+0000 7f3f522ef740  1 -- 172.21.15.71:0/1415225839 >> 172.21.15.71:0/1415225839 conn(0x1ca86f0 msgr2=0x1cecb20 unknown :-1 s=STATE_NONE l=0).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:47:28.715 INFO:teuthology.orchestra.run.smithi071.stdout:    }
, 2020-12-16T23:47:28.715 INFO:teuthology.orchestra.run.smithi071.stdout:}
, 2020-12-16T23:47:28.720+0000 7f3f43855700  1 -- 172.21.15.71:0/4106009391 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f3f2407ee50 msgr2=0x7f3f24081310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:48:03.443+0000 7fbfc679f700  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc00ba8b0 0x7fbfc00bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:48:03.443+0000 7fbfc679f700  1 --  --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0152200 con 0x7fbfa8004af0
, 2020-12-16T23:48:03.443+0000 7fbfc679f700  1 --  --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01524a0 con 0x7fbfc0155120

Ignoring malparsed backtrace: 2020-12-16T23:48:03.839+0000 7fe09e7fc700  1 --2- 172.21.15.71:0/1654353031 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7fe08c07ec40 0x7fe08c081100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:48:03.839+0000 7fe09e7fc700  1 -- 172.21.15.71:0/1654353031 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 7913+0+0 (secure 0 0 0) 0x7fe0a00a1000 con 0x7fe0b00c44d0
, 2020-12-16T23:48:03.839+0000 7fe0add9b700  1 --2- 172.21.15.71:0/1654353031 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7fe08c07ec40 0x7fe08c081100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0

Ignoring malparsed backtrace: 2020-12-16T23:48:23.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~49s
, 2020-12-16T23:48:24.695 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json
, 2020-12-16T23:48:24.875+0000 7f21fe11c700  1  Processor -- start

Ignoring malparsed backtrace: 2020-12-16T23:48:25.279+0000 7f6d5affd700  1 --2- 172.21.15.71:0/45557338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d48003950 0x7f6d48003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:48:25.279+0000 7f6d5affd700  1 --2- 172.21.15.71:0/45557338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d480044d0 0x7f6d48004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:48:25.279+0000 7f6d5affd700  1 --2- 172.21.15.71:0/45557338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d48004e40 0x7f6d4800b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect

Ignoring malparsed backtrace: 2020-12-16T23:48:45.674+0000 7fc709675700  1 --2- 172.21.15.71:0/1903795877 >> v2:172.21.15.71:3300/0 conn(0x7fc704151c80 0x7fc7040bfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:48:45.674+0000 7fc709675700  1 --2- 172.21.15.71:0/1903795877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc704065e10 0x7fc7040661f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:48:45.674+0000 7fc709675700  1 -- 172.21.15.71:0/1903795877 >> 172.21.15.71:0/1903795877 conn(0x7fc7040c9080 msgr2=0x7fc7040c9490 unknown :-1 s=STATE_NONE l=0).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:48:46.082+0000 7f747c05d700  1 -- 172.21.15.71:0/3861403944 shutdown_connections
, 2020-12-16T23:48:46.082+0000 7f747c05d700  1 -- 172.21.15.71:0/3861403944 wait complete.
, 2020-12-16T23:48:46.086+0000 7f747c05d700  1  Processor -- start

Ignoring malparsed backtrace: 2020-12-16T23:49:06.482+0000 7f2de1910700  1 --2- 172.21.15.71:0/3513354318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ddc064660 0x7f2ddc0bbfe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f2ddc158d80 tx=0x7f2dd8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0
, 2020-12-16T23:49:06.486+0000 7f2dc9ffb700  1 -- 172.21.15.71:0/3513354318 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8020070 con 0x7f2ddc064660
, 2020-12-16T23:49:06.486+0000 7f2dc9ffb700  1 -- 172.21.15.71:0/3513354318 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd801c420 con 0x7f2ddc064660

Ignoring malparsed backtrace: 2020-12-16T23:49:07.074+0000 7fde1f06b700  1 --2- 172.21.15.71:0/3636212869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde1814ed40 0x7fde1814d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:49:07.074+0000 7fde1f06b700  1 --2- 172.21.15.71:0/3636212869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde180bf4b0 0x7fde18165060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:49:07.074+0000 7fde1f06b700  1 --2- 172.21.15.71:0/3636212869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde1814e960 0x7fde1814f8e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:49:27.682+0000 7ffba179c700  1 -- 172.21.15.71:0/2933849975 >> v1:172.21.15.93:6789/0 conn(0x7ffb9c0be690 legacy=0x7ffb9c14d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:49:27.682+0000 7ffba179c700  1 -- 172.21.15.71:0/2933849975 >> v1:172.21.15.71:6789/0 conn(0x7ffb9c0ba8e0 legacy=0x7ffb9c0bc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:49:27.682+0000 7ffba0f9b700  1 -- 172.21.15.71:0/2933849975 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056178461 0 0) 0x7ffb9c0bd800 con 0x7ffb9c0be690

Ignoring malparsed backtrace: 2020-12-16T23:49:48.161+0000 7f0052e90700  1 --  --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c161f90 con 0x7f004c0be040
, 2020-12-16T23:49:48.161+0000 7f0052e90700  1 --  --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c162230 con 0x7f004c0bf890
, 2020-12-16T23:49:48.161+0000 7f005268f700  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f004c0bf890 0x7f004c15e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0

Ignoring malparsed backtrace: 2020-12-16T23:49:48.761 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":29,"fsid":"4311cddf-4de3-49c7-8fc3-ae06c23634c6","created":"2020-12-16T23:36:00.712075+0000","modified":"2020-12-16T23:46:11.367974+0000","last_up_change":"2020-12-16T23:46:10.352293+0000","last_in_change":"2020-12-16T23:36:08.576338+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-12-16T23:36:04.637405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-12-16T23:36:13.908642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-12-16T23:36:17.800920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-12-16T23:36:18.393830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"441c9f5c-f335-41cc-8faa-5710c98c457c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":16139},{"type":"v1","addr":"172.21.15.71:6801","nonce":16139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":16139},{"type":"v1","addr":"172.21.15.71:6803","nonce":16139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":16139},{"type":"v1","addr":"172.21.15.71:6807","nonce":16139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":16139},{"type":"v1","addr":"172.21.15.71:6805","nonce":16139}]},"public_addr":"172.21.15.71:6801/16139","cluster_addr":"172.21.15.71:6803/16139","heartbeat_back_addr":"172.21.15.71:6807/16139","heartbeat_front_addr":"172.21.15.71:6805/16139","state":["exists","up"]},{"osd":1,"uuid":"cffc7487-089d-42e9-855a-873593a2156c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":25,"down_at":28,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6823","nonce":17655},{"type":"v1","addr":"172.21.15.181:6825","nonce":17655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6827","nonce":17655},{"type":"v1","addr":"172.21.15.181:6829","nonce":17655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":17655},{"type":"v1","addr":"172.21.15.181:6833","nonce":17655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":17655},{"type":"v1","addr":"172.21.15.181:6831","nonce":17655}]},"public_addr":"172.21.15.181:6825/17655","cluster_addr":"172.21.15.181:6829/17655","heartbeat_back_addr":"172.21.15.181:6833/17655","heartbeat_front_addr":"172.21.15.181:6831/17655","state":["exists"]},{"osd":2,"uuid":"71af4538-a1d4-468e-84c9-cd47a0f4f6cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":17899},{"type":"v1","addr":"172.21.15.93:6809","nonce":17899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":17899},{"type":"v1","addr":"172.21.15.93:6811","nonce":17899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":17899},{"type":"v1","addr":"172.21.15.93:6815","nonce":17899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":17899},{"type":"v1","addr":"172.21.15.93:6813","nonce":17899}]},"public_addr":"172.21.15.93:6809/17899","cluster_addr":"172.21.15.93:6811/17899","heartbeat_back_addr":"172.21.15.93:6815/17899","heartbeat_front_addr":"172.21.15.93:6813/17899","state":["exists","up"]},{"osd":3,"uuid":"0383371c-15ac-4be7-8e97-2f8a3716731f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":16163},{"type":"v1","addr":"172.21.15.71:6821","nonce":16163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":16163},{"type":"v1","addr":"172.21.15.71:6824","nonce":16163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":16163},{"type":"v1","addr":"172.21.15.71:6831","nonce":16163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":16163},{"type":"v1","addr":"172.21.15.71:6828","nonce":16163}]},"public_addr":"172.21.15.71:6821/16163","cluster_addr":"172.21.15.71:6824/16163","heartbeat_back_addr":"172.21.15.71:6831/16163","heartbeat_front_addr":"172.21.15.71:6828/16163","state":["exists","up"]},{"osd":4,"uuid":"4a8b7a36-5bd3-4083-8cd7-e61a8033a030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":17638},{"type":"v1","addr":"172.21.15.181:6819","nonce":17638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":17638},{"type":"v1","addr":"172.21.15.181:6821","nonce":17638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":17638},{"type":"v1","addr":"172.21.15.181:6828","nonce":17638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":17638},{"type":"v1","addr":"172.21.15.181:6824","nonce":17638}]},"public_addr":"172.21.15.181:6819/17638","cluster_addr":"172.21.15.181:6821/17638","heartbeat_back_addr":"172.21.15.181:6828/17638","heartbeat_front_addr":"172.21.15.181:6824/17638","state":["exists","up"]},{"osd":5,"uuid":"a401c878-8c14-43b3-bc23-26c9361475c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":17900},{"type":"v1","addr":"172.21.15.93:6801","nonce":17900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":17900},{"type":"v1","addr":"172.21.15.93:6803","nonce":17900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":17900},{"type":"v1","addr":"172.21.15.93:6807","nonce":17900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":17900},{"type":"v1","addr":"172.21.15.93:6805","nonce":17900}]},"public_addr":"172.21.15.93:6801/17900","cluster_addr":"172.21.15.93:6803/17900","heartbeat_back_addr":"172.21.15.93:6807/17900","heartbeat_front_addr":"172.21.15.93:6805/17900","state":["exists","up"]},{"osd":6,"uuid":"f334cd2f-fee7-4279-899b-52aaecd160a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":16140},{"type":"v1","addr":"172.21.15.71:6809","nonce":16140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":16140},{"type":"v1","addr":"172.21.15.71:6811","nonce":16140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":16140},{"type":"v1","addr":"172.21.15.71:6815","nonce":16140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":16140},{"type":"v1","addr":"172.21.15.71:6813","nonce":16140}]},"public_addr":"172.21.15.71:6809/16140","cluster_addr":"172.21.15.71:6811/16140","heartbeat_back_addr":"172.21.15.71:6815/16140","heartbeat_front_addr":"172.21.15.71:6813/16140","state":["exists","up"]},{"osd":7,"uuid":"55eee285-ace0-4e95-b6ba-2065aa015b99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":17636},{"type":"v1","addr":"172.21.15.181:6803","nonce":17636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":17636},{"type":"v1","addr":"172.21.15.181:6805","nonce":17636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":17636},{"type":"v1","addr":"172.21.15.181:6809","nonce":17636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":17636},{"type":"v1","addr":"172.21.15.181:6807","nonce":17636}]},"public_addr":"172.21.15.181:6803/17636","cluster_addr":"172.21.15.181:6805/17636","heartbeat_back_addr":"172.21.15.181:6809/17636","heartbeat_front_addr":"172.21.15.181:6807/17636","state":["exists","up"]},{"osd":8,"uuid":"68bb20e7-9900-4a36-a229-df3cb7f3367c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":17920},{"type":"v1","addr":"172.21.15.93:6825","nonce":17920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":17920},{"type":"v1","addr":"172.21.15.93:6827","nonce":17920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":17920},{"type":"v1","addr":"172.21.15.93:6831","nonce":17920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":17920},{"type":"v1","addr":"172.21.15.93:6829","nonce":17920}]},"public_addr":"172.21.15.93:6825/17920","cluster_addr":"172.21.15.93:6827/17920","heartbeat_back_addr":"172.21.15.93:6831/17920","heartbeat_front_addr":"172.21.15.93:6829/17920","state":["exists","up"]},{"osd":9,"uuid":"29ae7ea6-4eb6-40cc-a30b-b09b450d8fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":16167},{"type":"v1","addr":"172.21.15.71:6817","nonce":16167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":16167},{"type":"v1","addr":"172.21.15.71:6820","nonce":16167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6827","nonce":16167},{"type":"v1","addr":"172.21.15.71:6829","nonce":16167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6823","nonce":16167},{"type":"v1","addr":"172.21.15.71:6825","nonce":16167}]},"public_addr":"172.21.15.71:6817/16167","cluster_addr":"172.21.15.71:6820/16167","heartbeat_back_addr":"172.21.15.71:6829/16167","heartbeat_front_addr":"172.21.15.71:6825/16167","state":["exists","up"]},{"osd":10,"uuid":"a96313d2-2678-4a49-87b1-d09d50f7e6d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":17637},{"type":"v1","addr":"172.21.15.181:6811","nonce":17637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":17637},{"type":"v1","addr":"172.21.15.181:6813","nonce":17637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":17637},{"type":"v1","addr":"172.21.15.181:6817","nonce":17637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":17637},{"type":"v1","addr":"172.21.15.181:6815","nonce":17637}]},"public_addr":"172.21.15.181:6811/17637","cluster_addr":"172.21.15.181:6813/17637","heartbeat_back_addr":"172.21.15.181:6817/17637","heartbeat_front_addr":"172.21.15.181:6815/17637","state":["exists","up"]},{"osd":11,"uuid":"78cdd411-37a8-44cb-bbc7-1353c587c1e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":17901},{"type":"v1","addr":"172.21.15.93:6817","nonce":17901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":17901},{"type":"v1","addr":"172.21.15.93:6819","nonce":17901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":17901},{"type":"v1","addr":"172.21.15.93:6823","nonce":17901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":17901},{"type":"v1","addr":"172.21.15.93:6821","nonce":17901}]},"public_addr":"172.21.15.93:6817/17901","cluster_addr":"172.21.15.93:6819/17901","heartbeat_back_addr":"172.21.15.93:6823/17901","heartbeat_front_addr":"172.21.15.93:6821/17901","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.352006+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.433812+0000","dead_epoch":28},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.098480+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.523314+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.397213+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.032555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.377365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.055008+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.390152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.472883+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.394472+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-12-16T23:36:07.168997+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}}
, 2020-12-16T23:49:48.757+0000 7f6b97aeb700  1 -- 172.21.15.71:0/1994308149 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f6b7007ec40 msgr2=0x7f6b70081100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:49:48.757+0000 7f6b97aeb700  1 --2- 172.21.15.71:0/1994308149 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f6b7007ec40 0x7f6b70081100 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f6b8000b190 tx=0x7f6b80016000).stop

Ignoring malparsed backtrace: 2020-12-16T23:50:09.385+0000 7f04c3fff700  1 --1-  >> v1:172.21.15.93:6789/0 conn(0x7f04c409fde0 0x7f04c409eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.71:36752/0 (socket says 172.21.15.71:36752)
, 2020-12-16T23:50:09.385+0000 7f04c3fff700  1 -- 172.21.15.71:0/4248351256 learned_addr learned my addr 172.21.15.71:0/4248351256 (peer_addr_for_me v1:172.21.15.71:0/0)
, 2020-12-16T23:50:09.385+0000 7f04c37fe700  1 --1- 172.21.15.71:0/4248351256 >> v1:172.21.15.181:6789/0 conn(0x7f04c40a01c0 0x7f04c40c3680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.71:42774/0 (socket says 172.21.15.71:42774)

Ignoring malparsed backtrace: 2020-12-16T23:50:29.773+0000 7f7e9a59c700  1 --2- 172.21.15.71:0/1359180846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e9c14d7f0 0x7f7e9c0bbaf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f7e8c00b980 tx=0x7f7e8c006de0).ready entity=mon.1 client_cookie=21689bdcafff88ce server_cookie=0 in_seq=0 out_seq=0
, 2020-12-16T23:50:29.777+0000 7f7e9959a700  1 -- 172.21.15.71:0/1359180846 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e900014e0 con 0x7f7e9c14d7f0
, 2020-12-16T23:50:29.777+0000 7f7e9959a700  1 -- 172.21.15.71:0/1359180846 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c011400 con 0x7f7e9c14d7f0

Ignoring malparsed backtrace: 2020-12-16T23:50:30.173+0000 7ffb03fff700  1 -- 172.21.15.71:0/448984728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c0bf890 msgr2=0x7ffb0c150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:50:30.173+0000 7ffb03fff700  1 --2- 172.21.15.71:0/448984728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c0bf890 0x7ffb0c150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:50:30.173+0000 7ffb03fff700  1 -- 172.21.15.71:0/448984728 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb0c067bb0 msgr2=0x7ffb0c0bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:50:50.745+0000 7f7f05c44700  1 -- 172.21.15.71:0/3297695288 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f7ee007ec40 msgr2=0x7f7ee0081100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:50:50.745+0000 7f7f05c44700  1 --2- 172.21.15.71:0/3297695288 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7f7ee007ec40 0x7f7ee0081100 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f7eec006a00 tx=0x7f7eec008040).stop
, 2020-12-16T23:50:50.745+0000 7f7f05c44700  1 -- 172.21.15.71:0/3297695288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f00080140 msgr2=0x7f7f001535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:51:11.320+0000 7f385d433700  1 --  --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38580bf220 con 0x7f38580bfea0
, 2020-12-16T23:51:11.320+0000 7f385d433700  1 --  --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38580bf4c0 con 0x7f3858151000
, 2020-12-16T23:51:11.320+0000 7f38567fc700  1 --2-  >> v2:172.21.15.93:3300/0 conn(0x7f38580bfea0 0x7f3858060390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0

Ignoring malparsed backtrace: 2020-12-16T23:51:11.712+0000 7f8e80dc7700  1  Processor -- start
, 2020-12-16T23:51:11.712+0000 7f8e80dc7700  1 --  start start
, 2020-12-16T23:51:11.712+0000 7f8e80dc7700  1 --2-  >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8e7c0ba8e0 0x7f8e7c1645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect

Ignoring malparsed backtrace: 2020-12-16T23:51:32.096+0000 7ff4577fe700  1 -- 172.21.15.71:0/420651198 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45000b8a0 con 0x7ff460064260
, 2020-12-16T23:51:32.096+0000 7ff4577fe700  1 -- 172.21.15.71:0/420651198 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45000ba00 con 0x7ff460064260
, 2020-12-16T23:51:32.096+0000 7ff4649f2700  1 -- 172.21.15.71:0/420651198 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4601612e0 con 0x7ff460064260

Ignoring malparsed backtrace: 2020-12-16T23:51:32.648+0000 7fc50db63700  1 --2- 172.21.15.71:0/4014857115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc50814de90 0x7fc508149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:51:32.648+0000 7fc50db63700  1 --2- 172.21.15.71:0/4014857115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc508069430 0x7fc508162230 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:51:32.648+0000 7fc50db63700  1 --2- 172.21.15.71:0/4014857115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc50814a7c0 0x7fc5080b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:51:53.244+0000 7f3539ebf700  1 -- 172.21.15.71:0/27057049 >> v1:172.21.15.93:6789/0 conn(0x7f3534151c90 legacy=0x7f35340bf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:51:53.244+0000 7f3539ebf700  1 -- 172.21.15.71:0/27057049 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340802e0 con 0x7f35340c44d0
, 2020-12-16T23:51:53.248+0000 7f3539ebf700  1 --2- 172.21.15.71:0/27057049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35340c44d0 0x7f35340c48b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f353000b9e0 tx=0x7f3530006ea0).ready entity=mon.1 client_cookie=22b2912a67e71b07 server_cookie=0 in_seq=0 out_seq=0

Ignoring malparsed backtrace: 2020-12-16T23:52:13.632+0000 7fb7e2276700  1 --2-  >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7dc0b9210 0x7fb7dc0642e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.71:49002/0 (socket says 172.21.15.71:49002)
, 2020-12-16T23:52:13.632+0000 7fb7e2276700  1 -- 172.21.15.71:0/1898780999 learned_addr learned my addr 172.21.15.71:0/1898780999 (peer_addr_for_me v2:172.21.15.71:0/0)
, 2020-12-16T23:52:13.632+0000 7fb7e2a77700  1 --2- 172.21.15.71:0/1898780999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7dc0bc160 0x7fb7dc14b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0

Ignoring malparsed backtrace: 2020-12-16T23:52:14.032+0000 7f2d20eee700  1 --2- 172.21.15.71:0/3876468151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d08003510 0x7f2d1c15f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop
, 2020-12-16T23:52:14.032+0000 7f2d20eee700  1 -- 172.21.15.71:0/3876468151 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d1c0b2150 msgr2=0x7f2d1c0631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:52:14.032+0000 7f2d20eee700  1 --2- 172.21.15.71:0/3876468151 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d1c0b2150 0x7f2d1c0631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:52:34.631+0000 7fd966512700  1 -- 172.21.15.71:0/384377088 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7fd94007ec70 msgr2=0x7fd940081130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
, 2020-12-16T23:52:34.631+0000 7fd966512700  1 --2- 172.21.15.71:0/384377088 >> [v2:172.21.15.181:6800/17530,v1:172.21.15.181:6801/17530] conn(0x7fd94007ec70 0x7fd940081130 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fd95c005590 tx=0x7fd95c009040).stop
, 2020-12-16T23:52:34.631+0000 7fd966512700  1 -- 172.21.15.71:0/384377088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd960152670 msgr2=0x7fd960164620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down

Ignoring malparsed backtrace: 2020-12-16T23:52:36.481 DEBUG:teuthology.orchestra.run:got remote process result: 1
, 2020-12-16T23:52:38.489 INFO:tasks.ceph.osd.0:Sent signal 15
, 2020-12-16T23:52:38.491 INFO:tasks.ceph.osd.3:Sent signal 15

Ignoring malparsed backtrace: 2020-12-16T23:52:58.203+0000 7f388ffff700  1 --  >> v1:172.21.15.93:6789/0 conn(0x7f38700033b0 legacy=0x7f3870005800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.93:6789/0
, 2020-12-16T23:52:58.203+0000 7f388f7fe700  1 --2-  >> v2:172.21.15.181:3300/0 conn(0x7f3870007030 0x7f38700094f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.200000
, 2020-12-16T23:52:58.203+0000 7f3894ce8700  1 --  >> v2:172.21.15.71:3300/0 conn(0x7f3870009b90 msgr2=0x7f387000c050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.71:3300/0

Ignoring malparsed backtrace: 2020-12-16T23:53:08.619+0000 7f3894ce8700  1 --2-  >> v2:172.21.15.71:3300/0 conn(0x7f3870009500 0x7f3870006750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000
, 2020-12-16T23:53:10.207+0000 7f388dffb700  1 --  >> v2:172.21.15.71:3300/0 conn(0x7f3870009500 msgr2=0x7f3870006750 unknown :-1 s=STATE_CONNECTING l=0).mark_down
, 2020-12-16T23:53:10.207+0000 7f388dffb700  1 --2-  >> v2:172.21.15.71:3300/0 conn(0x7f3870009500 0x7f3870006750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:53:19.415+0000 7f3894ce8700  1 --  >> v1:172.21.15.93:6789/0 conn(0x7f3870015780 legacy=0x7f387001b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.93:6789/0
, 2020-12-16T23:53:19.415+0000 7f388f7fe700  1 --  >> v1:172.21.15.71:6789/0 conn(0x7f3870012380 legacy=0x7f3870017ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.71:6789/0
, 2020-12-16T23:53:19.415+0000 7f388ffff700  1 --2-  >> v2:172.21.15.181:3300/0 conn(0x7f3870018840 0x7f3870018c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.400000

Ignoring malparsed backtrace: 2020-12-16T23:53:31.211+0000 7f388dffb700  1 --2-  >> v2:172.21.15.71:3300/0 conn(0x7f3870004c90 0x7f3870011ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect
, 2020-12-16T23:53:31.211+0000 7f3894ce8700  1 --  reap_dead start
, 2020-12-16T23:53:31.211+0000 7f388dffb700  1 --  --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38900c1920 con 0x7f38700094f0

Ignoring malparsed backtrace: 2020-12-16T23:53:40.415+0000 7f3894ce8700  1 --  >> v1:172.21.15.181:6789/0 conn(0x7f3870017b90 legacy=0x7f3870003c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.181:6789/0
, 2020-12-16T23:53:40.415+0000 7f388f7fe700  1 --  >> v1:172.21.15.93:6789/0 conn(0x7f38700094f0 legacy=0x7f38700153a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.93:6789/0
, 2020-12-16T23:53:40.819+0000 7f388ffff700  1 --  >> v1:172.21.15.71:6789/0 conn(0x7f3870003790 legacy=0x7f387000cc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.71:6789/0

Ignoring malparsed backtrace: 2020-12-16T23:53:52.419+0000 7f388f7fe700  1 --  >> v1:172.21.15.93:6789/0 conn(0x7f387000cc50 legacy=0x7f3870018770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.93:6789/0
, 2020-12-16T23:53:52.419+0000 7f388ffff700  1 --  >> v2:172.21.15.93:3300/0 conn(0x7f38700054f0 msgr2=0x7f3870011a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.93:3300/0
, 2020-12-16T23:53:52.419+0000 7f3894ce8700  1 --  >> v2:172.21.15.71:3300/0 conn(0x7f387000af80 msgr2=0x7f387000b390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.71:3300/0

Ignoring malparsed backtrace: 2020-12-16T23:54:02.626+0000 7f388ffff700  1 --2-  >> v2:172.21.15.71:3300/0 conn(0x7f387000b880 0x7f387000cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000
, 2020-12-16T23:54:04.214+0000 7f388dffb700  1 --  >> v2:172.21.15.93:3300/0 conn(0x7f38700054b0 msgr2=0x7f387000b390 unknown :-1 s=STATE_CONNECTING l=0).mark_down
, 2020-12-16T23:54:04.214+0000 7f388dffb700  1 --2-  >> v2:172.21.15.93:3300/0 conn(0x7f38700054b0 0x7f387000b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:54:13.826+0000 7f3894ce8700  1 --  >> v1:172.21.15.71:6789/0 conn(0x7f387000cc50 legacy=0x7f387001d8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.71:6789/0
, 2020-12-16T23:54:13.826+0000 7f388f7fe700  1 --2-  >> v2:172.21.15.71:3300/0 conn(0x7f387000b880 0x7f3870011a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000
, 2020-12-16T23:54:13.826+0000 7f388ffff700  1 --  >> v1:172.21.15.181:6789/0 conn(0x7f3870017b90 legacy=0x7f3870003bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.181:6789/0

Ignoring malparsed backtrace: 2020-12-16T23:54:25.218+0000 7f388dffb700  1 --  --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38900be6b0 con 0x7f387000cb00
, 2020-12-16T23:54:25.218+0000 7f388ffff700  1 --  >> v1:172.21.15.71:6789/0 conn(0x7f387000b880 legacy=0x7f3870003b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.71:6789/0
, 2020-12-16T23:54:25.218+0000 7f388f7fe700  1 --  >> v2:172.21.15.93:3300/0 conn(0x7f387000cb00 msgr2=0x7f387000b360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.93:3300/0

Ignoring malparsed backtrace: 2020-12-16T23:54:35.634+0000 7f3894ce8700  1 --2-  >> v2:172.21.15.181:3300/0 conn(0x7f38700182a0 0x7f3870011a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 1.600000
, 2020-12-16T23:54:37.222+0000 7f388dffb700  1 --  >> v2:172.21.15.181:3300/0 conn(0x7f38700182a0 msgr2=0x7f3870011a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down
, 2020-12-16T23:54:37.222+0000 7f388dffb700  1 --2-  >> v2:172.21.15.181:3300/0 conn(0x7f38700182a0 0x7f3870011a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop

Ignoring malparsed backtrace: 2020-12-16T23:54:46.834+0000 7f3894ce8700  1 --  >> v2:172.21.15.181:3300/0 conn(0x7f3870011340 msgr2=0x7f3870011750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.181:3300/0
, 2020-12-16T23:54:46.834+0000 7f3894ce8700  1 --2-  >> v2:172.21.15.181:3300/0 conn(0x7f3870011340 0x7f3870011750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._fault waiting 0.800000
, 2020-12-16T23:54:46.834+0000 7f388ffff700  1 --  >> v1:172.21.15.93:6789/0 conn(0x7f387000b820 legacy=0x7f3870017b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.93:6789/0

Ignoring malparsed backtrace: 2020-12-16T23:54:55.095 DEBUG:tasks.ceph.osd.0:waiting for process to exit
, 2020-12-16T23:54:55.095 INFO:teuthology.orchestra.run:waiting for 300
, 2020-12-16T23:54:55.096 INFO:tasks.ceph.osd.0:Stopped

Ignoring malparsed backtrace:   File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 183, in _raise_for_status
,     node=self.hostname, label=self.label
, teuthology.exceptions.CommandFailedError: Command failed on smithi071 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json'

Ignoring malparsed backtrace: 2020-12-16T23:54:58.581 DEBUG:teuthology.orchestra.run.smithi093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8
, 2020-12-16T23:54:58.764 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi093.front.sepia.ceph.com
, 2020-12-16T23:54:58.765 DEBUG:teuthology.orchestra.run.smithi093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11

Ignoring malparsed backtrace:     'stdout': StringIO()}).stdout.getvalue()
,   File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201214.211711/qa/tasks/ceph_manager.py", line 1347, in run_cluster_cmd
,     return self.controller.run(**kwargs)

Ignoring malparsed backtrace:     'stdout': StringIO()}).stdout.getvalue()
,   File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201214.211711/qa/tasks/ceph_manager.py", line 1347, in run_cluster_cmd
,     return self.controller.run(**kwargs)

Not checking for backtrace from smithi181:osd.1 log, too large (1390214435)