Found 51 jobs Missing teuthology log /home/teuthworker/archive/pdonnell-2023-05-21_14:16:58-fs-wip-pdonnell-testing-20230519.164114-distro-default-smithi/7281776/teuthology.log Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Ignoring malparsed backtrace: ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x128) [0x7fc8e6c4d257] , 2: /usr/lib/ceph/libceph-common.so.2(+0x31e413) [0x7fc8e6c4d413] Ignoring malparsed backtrace: 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 -- start start , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff9f803adc0 0x7ff9f80c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:17.744+0000 7ffa050bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9f803b190 0x7ff9f8164990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c14a520 con 0x7f7a8c06df20 , 2023-05-21T14:57:18.056+0000 7f7a9153d700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c13fc90 con 0x7f7a8c13f8c0 , 2023-05-21T14:57:18.056+0000 7f7a90d3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a8c13d2c0 0x7f7a8c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:57:18.384+0000 7f69fb7fe700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69fc1ddef0 0x7f69fc0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 -- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 msgr2=0x7f69fc1db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:18.384+0000 7f69fa7fc700 1 --2- 172.21.15.61:0/4163968434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f69e8003bb0 0x7f69fc1db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03bc06bcd0 0x7f03bc0b4b60 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c520 tx=0x7f03ac00c830 comp rx=0 tx=0).stop , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 -- 172.21.15.61:0/52808872 shutdown_connections , 2023-05-21T14:57:18.868+0000 7f03bbfff700 1 --2- 172.21.15.61:0/52808872 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f03a8066320 0x7f03a80687d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:19.320+0000 7f2c9c92d700 1 -- 172.21.15.61:0/984546343 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980b5fa0 con 0x7f2c980b3c70 , 2023-05-21T14:57:19.320+0000 7f2c977fe700 1 --2- 172.21.15.61:0/984546343 >> v2:172.21.15.103:3300/0 conn(0x7f2c980b73b0 0x7f2c98145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! Ignoring malparsed backtrace: 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.609 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", Ignoring malparsed backtrace: 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:19.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", Ignoring malparsed backtrace: 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", , 2023-05-21T14:57:19.768 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:19.789 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", Ignoring malparsed backtrace: 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:19.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:19.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:19.828 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { , 2023-05-21T14:57:19.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.889 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:19.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:19.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:19.928 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:19.929 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", , 2023-05-21T14:57:19.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:19.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:19.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", Ignoring malparsed backtrace: 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:19.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.029 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.068 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.089 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:20.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:20.110 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", Ignoring malparsed backtrace: 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.149 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.168 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.189 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.221 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:20.278 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:20.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.318 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.345 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.393 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", , 2023-05-21T14:57:20.394 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", , 2023-05-21T14:57:20.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", Ignoring malparsed backtrace: 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.435 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", Ignoring malparsed backtrace: 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.454 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.473 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.512 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { , 2023-05-21T14:57:20.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { Ignoring malparsed backtrace: 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:20.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.564 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:20.586 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", , 2023-05-21T14:57:20.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", Ignoring malparsed backtrace: 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { , 2023-05-21T14:57:20.636 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:20.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:20.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", , 2023-05-21T14:57:20.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:20.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", , 2023-05-21T14:57:20.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:20.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:20.880 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", , 2023-05-21T14:57:20.881 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", Ignoring malparsed backtrace: 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:20.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:20.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:20.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:20.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:20.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "alertmanager_web_password": { , 2023-05-21T14:57:21.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alertmanager_web_password", , 2023-05-21T14:57:21.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", Ignoring malparsed backtrace: 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.099 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", , 2023-05-21T14:57:21.121 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_web_password", , 2023-05-21T14:57:21.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", Ignoring malparsed backtrace: 2023-05-21T14:57:21.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", , 2023-05-21T14:57:21.170 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", , 2023-05-21T14:57:21.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.209 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.210 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { , 2023-05-21T14:57:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", Ignoring malparsed backtrace: 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", , 2023-05-21T14:57:21.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], Ignoring malparsed backtrace: 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "info", , 2023-05-21T14:57:21.396 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" , 2023-05-21T14:57:21.397 INFO:teuthology.orchestra.run.smithi061.stdout: ], Ignoring malparsed backtrace: 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", , 2023-05-21T14:57:21.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, Ignoring malparsed backtrace: 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:21.520 INFO:teuthology.orchestra.run.smithi061.stdout: "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ Ignoring malparsed backtrace: 2023-05-21T14:57:21.588 INFO:teuthology.orchestra.run.smithi061.stdout: { , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", , 2023-05-21T14:57:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, Ignoring malparsed backtrace: 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { , 2023-05-21T14:57:21.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", , 2023-05-21T14:57:21.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:21.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", Ignoring malparsed backtrace: 2023-05-21T14:57:21.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { , 2023-05-21T14:57:21.730 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", Ignoring malparsed backtrace: 2023-05-21T14:57:21.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", , 2023-05-21T14:57:21.778 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, Ignoring malparsed backtrace: 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", , 2023-05-21T14:57:21.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:21.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: ], , 2023-05-21T14:57:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", Ignoring malparsed backtrace: 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, , 2023-05-21T14:57:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", , 2023-05-21T14:57:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { Ignoring malparsed backtrace: 2023-05-21T14:57:21.962 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", , 2023-05-21T14:57:21.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", Ignoring malparsed backtrace: 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:21.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { , 2023-05-21T14:57:22.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", Ignoring malparsed backtrace: 2023-05-21T14:57:22.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.074 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.107 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.108 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", , 2023-05-21T14:57:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], , 2023-05-21T14:57:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", Ignoring malparsed backtrace: 2023-05-21T14:57:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], , 2023-05-21T14:57:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] Ignoring malparsed backtrace: 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { , 2023-05-21T14:57:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", Ignoring malparsed backtrace: 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", , 2023-05-21T14:57:22.259 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], Ignoring malparsed backtrace: 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", , 2023-05-21T14:57:22.290 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", Ignoring malparsed backtrace: 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ , 2023-05-21T14:57:22.320 INFO:teuthology.orchestra.run.smithi061.stdout: "", , 2023-05-21T14:57:22.321 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", Ignoring malparsed backtrace: 2023-05-21T14:57:22.358 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", , 2023-05-21T14:57:22.359 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", Ignoring malparsed backtrace: 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, , 2023-05-21T14:57:22.526 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 142432, , 2023-05-21T14:57:22.527 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 116784 Ignoring malparsed backtrace: 2023-05-21T14:57:22.557 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, , 2023-05-21T14:57:22.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { , 2023-05-21T14:57:22.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2142, Ignoring malparsed backtrace: 2023-05-21T14:57:22.728 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 13359, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 6, , 2023-05-21T14:57:22.729 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { , 2023-05-21T14:57:22.749 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, Ignoring malparsed backtrace: 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { , 2023-05-21T14:57:22.769 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 13240, , 2023-05-21T14:57:22.770 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 24.870964326, Ignoring malparsed backtrace: 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 759, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 108650, , 2023-05-21T14:57:22.794 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, Ignoring malparsed backtrace: 2023-05-21T14:57:22.814 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 109409, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 511527310, , 2023-05-21T14:57:22.815 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { Ignoring malparsed backtrace: 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc41de8f0 0x7f0fc413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4143170 con 0x7f0fc40b0aa0 , 2023-05-21T14:57:22.940+0000 7f0fca0ff700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc413d650 con 0x7f0fc40b8070 Ignoring malparsed backtrace: 2023-05-21T14:57:23.296+0000 7f0731ffb700 1 -- 172.21.15.61:0/3360551182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0738085b10 con 0x7f073c0b2320 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:23.296+0000 7f07417b8700 1 --2- 172.21.15.61:0/3360551182 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f0720066320 0x7f07200687d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f072c005930 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:23.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 Processor -- start , 2023-05-21T14:57:23.984+0000 7f46ad39c700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.324+0000 7fb21359e700 1 -- 172.21.15.61:0/2288009770 wait complete. , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 Processor -- start , 2023-05-21T14:57:25.328+0000 7fb21359e700 1 -- start start Ignoring malparsed backtrace: 2023-05-21T14:57:25.504+0000 7f8a42436700 1 -- 172.21.15.61:0/2913294185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34009a70 con 0x7f8a3c13cbf0 , 2023-05-21T14:57:25.504+0000 7f8a41c35700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a3c06f0c0 0x7f8a3c13f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! , 2023-05-21T14:57:25.504+0000 7f8a42436700 1 --2- 172.21.15.61:0/2913294185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a3c13cbf0 0x7f8a3c135b00 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c070bd0 tx=0x7f8a34004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:25.836+0000 7f5531ffb700 1 -- 172.21.15.61:0/220378656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004020 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340b5ed0 con 0x7f5534142b40 , 2023-05-21T14:57:25.836+0000 7f553a0a5700 1 -- 172.21.15.61:0/220378656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f553404e2b0 con 0x7f5534142b40 Ignoring malparsed backtrace: 2023-05-21T14:57:26.184+0000 7fec93fff700 1 -- 172.21.15.61:0/1656643296 wait complete. , instructed osd(s) 1 to deep-scrub , 2023-05-21T14:57:26.235 INFO:tasks.ceph:Scrubbing osd.2 Ignoring malparsed backtrace: 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> v2:172.21.15.103:3300/0 conn(0x7f93540b82e0 0x7f9354143050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:26.504+0000 7f935259c700 1 -- 172.21.15.61:0/1826824424 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b86b0 con 0x7f935413c770 , 2023-05-21T14:57:26.504+0000 7f935259c700 1 --2- 172.21.15.61:0/1826824424 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f935413c770 0x7f935413cbc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9354073be0 tx=0x7f934400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4524e92de0970f server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 learned_addr learned my addr 172.21.15.61:0/2991578969 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:57:26.828+0000 7f3cfdd9b700 1 --2- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 0x7f3d0013ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:26.828+0000 7f3cfed9d700 1 -- 172.21.15.61:0/2991578969 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d0013fd00 msgr2=0x7f3d0013ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 85658+0+0 (secure 0 0 0) 0x7fa0e00036c0 con 0x7fa0ec0b82e0 , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 --2- 172.21.15.61:0/2427253240 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa0d00662f0 0x7fa0d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect , 2023-05-21T14:57:27.000+0000 7fa0e97fa700 1 -- 172.21.15.61:0/2427253240 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fa0e00834e0 con 0x7fa0ec0b82e0 Ignoring malparsed backtrace: 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 Processor -- start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 -- start start , 2023-05-21T14:57:27.484+0000 7fb32aa95700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb3240b5d10 0x7fb3240b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2e0006d1a0 0x7f2e00143790 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009880 tx=0x7f2dfc009b90 comp rx=0 tx=0).stop , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 -- 172.21.15.61:0/1723469075 shutdown_connections , 2023-05-21T14:57:27.828+0000 7f2e07012700 1 --2- 172.21.15.61:0/1723469075 >> v2:172.21.15.103:3300/0 conn(0x7f2e000b07b0 0x7f2e000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 msgr2=0x7ff5a0136710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 --2- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5a01444f0 0x7ff5a0136710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:57:28.000+0000 7ff59e59c700 1 -- 172.21.15.61:0/973479930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5a00b82e0 msgr2=0x7ff5a01de740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:28.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2023-05-21T14:57:28.383 INFO:teuthology.orchestra.run.smithi061.stdout:} Ignoring malparsed backtrace: 2023-05-21T14:57:28.844+0000 7f4f9effd700 1 -- 172.21.15.61:0/1027835717 learned_addr learned my addr 172.21.15.61:0/1027835717 (peer_addr_for_me v1:172.21.15.61:0/0) , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.103:6789/0 conn(0x7f4f98007fe0 legacy=0x7f4f980cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:57:28.844+0000 7f4f9ffff700 1 -- 172.21.15.61:0/1027835717 >> v1:172.21.15.61:6789/0 conn(0x7f4f98036a80 legacy=0x7f4f980c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down Ignoring malparsed backtrace: 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041dbc50 con 0x7f180413c820 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413ddb0 con 0x7f180406f0c0 , 2023-05-21T14:57:29.012+0000 7f180987f700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e090 con 0x7f180413cbf0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.328+0000 7f4994ff9700 1 -- 172.21.15.61:0/978013853 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f4990084820 con 0x7f49980661a0 , 2023-05-21T14:57:29.328+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:57:29.332+0000 7f49977fe700 1 --2- 172.21.15.61:0/978013853 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f49840662f0 0x7f49840687a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49981d87b0 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 Ignoring malparsed backtrace: 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.620 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-05-21T14:45:20.518764+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=20, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) , 2023-05-21T14:57:29.621 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-05-21T14:45:19.514569+0000 time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=45, tm_sec=19, tm_wday=6, tm_yday=141, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=5, tm_mday=21, tm_hour=14, tm_min=57, tm_sec=24, tm_wday=6, tm_yday=141, tm_isdst=0) Ignoring malparsed backtrace: 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 >> 172.21.15.61:0/2883262753 conn(0x7f174806ab10 msgr2=0x7f174806b800 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 shutdown_connections , 2023-05-21T14:57:49.955+0000 7f174dbe7700 1 -- 172.21.15.61:0/2883262753 wait complete. Ignoring malparsed backtrace: 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f1894009e50 tx=0x7f189400b330 comp rx=0 tx=0).stop , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 -- 172.21.15.61:0/1652788197 shutdown_connections , 2023-05-21T14:58:10.123+0000 7f18a21f4700 1 --2- 172.21.15.61:0/1652788197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f189c13be50 0x7f189c13c2c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 Processor -- start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 -- start start , 2023-05-21T14:58:30.498+0000 7f9f4c819700 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9f480b4f00 0x7f9f48145560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect Ignoring malparsed backtrace: 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3e2c0b3c70 0x7f3e2c13fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.61:48968/0 (socket says 172.21.15.61:48968) , 2023-05-21T14:58:50.854+0000 7f3e322f9700 1 -- 172.21.15.61:0/142222863 learned_addr learned my addr 172.21.15.61:0/142222863 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:58:50.854+0000 7f3e31af8700 1 --2- 172.21.15.61:0/142222863 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3e2c0b38a0 0x7f3e2c0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 Ignoring malparsed backtrace: 2023-05-21T14:59:11.326+0000 7fc522ffd700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc50c003a20 0x7fc50c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 -- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 msgr2=0x7fc50c005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down , 2023-05-21T14:59:11.326+0000 7fc5227fc700 1 --2- 172.21.15.61:0/157564906 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc52406c680 0x7fc50c005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c0b38a0 0x7f4b5c1da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34090/0 (socket says 172.21.15.61:34090) , 2023-05-21T14:59:11.690+0000 7f4b5ad9d700 1 -- 172.21.15.61:0/1323728933 learned_addr learned my addr 172.21.15.61:0/1323728933 (peer_addr_for_me v2:172.21.15.61:0/0) Ignoring malparsed backtrace: 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea4813c450 0x7fea4813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 -- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 msgr2=0x7fea481d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.014+0000 7fea4c922700 1 --2- 172.21.15.61:0/1589699905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea30003fa0 0x7fea481d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7fa554066320 0x7fa5540687d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006fd0 tx=0x7fa55c008040 comp rx=0 tx=0).stop , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 -- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 msgr2=0x7fa5680b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:12.554+0000 7fa56cf88700 1 --2- 172.21.15.61:0/1587232970 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa568149150 0x7fa5680b6a20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cb80 tx=0x7fa56000ce90 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 learned_addr learned my addr 172.21.15.61:0/3998261842 (peer_addr_for_me v2:172.21.15.61:0/0) , 2023-05-21T14:59:12.990+0000 7f0303fff700 1 --2- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 0x7f030c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 , 2023-05-21T14:59:12.990+0000 7f030b7fe700 1 -- 172.21.15.61:0/3998261842 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f030c13cbf0 msgr2=0x7f030c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down Ignoring malparsed backtrace: 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40ab4d0 con 0x7fa4a406a800 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41cff20 con 0x7fa4a40b1280 , 2023-05-21T14:59:13.354+0000 7fa4a359e700 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40aae80 con 0x7fa4a41469f0 Ignoring malparsed backtrace: 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.103:6832/2528111820,v1:172.21.15.103:6833/2528111820] conn(0x7f6e54066320 0x7f6e540687d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e68001800 tx=0x7f6e68009040 comp rx=0 tx=0).stop , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 -- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 msgr2=0x7f6e7413fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down , 2023-05-21T14:59:14.910+0000 7f6e7359e700 1 --2- 172.21.15.61:0/1017808878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6e74142c40 0x7f6e7413fb40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00edd0 tx=0x7f6e6c00ee00 comp rx=0 tx=0).stop Ignoring malparsed backtrace: 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 --2- 172.21.15.61:0/3152108652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5d4164bb0 0x7ff5d40c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 >> 172.21.15.61:0/3152108652 conn(0x7ff5d403e920 msgr2=0x7ff5d40c25d0 unknown :-1 s=STATE_NONE l=0).mark_down , 2023-05-21T14:59:15.910+0000 7ff5e3237700 1 -- 172.21.15.61:0/3152108652 shutdown_connections Ignoring malparsed backtrace: 2023-05-21T14:59:25.647 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , 2023-05-21T14:59:25.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq , gzip: /var/log/ceph/valgrind/*.gz: No such file or directory Ignoring malparsed backtrace: /var/log/ceph/ceph-client.admin.40180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63008.log.gz , 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz , gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log Ignoring malparsed backtrace: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz , /var/log/ceph/ceph-client.admin.40392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz , -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log Ignoring malparsed backtrace: 94.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz , 93.9% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz , Found 25 distinct failure reasons Failure: Test failure: test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) 1 jobs: ['7281797'] suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/openfiletable}'] Timeout 3h running clone.client.1/qa/workunits/suites/dbench.sh 2 jobs: ['7281769', '7281798'] suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'ms-die-on-skipped}', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'tasks/kernel_cfuse_workunits_dbench_iozone}'] suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/stock/{k-stock', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/kernel_cfuse_workunits_dbench_iozone}'] Failure: Command failed on smithi148 with status 2: 'TESTDIR=/home/ubuntu/cephtest bash -s' 3 jobs: ['7281780', '7281799', '7281775'] suites intersection: ['1-ceph', '2-logrotate}', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{ignorelist_health'] suites union: ['1-ceph', '2-logrotate}', 'clusters/1-mds-2-client', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/ior-shared-file}', 'tasks/mdtest}'] Failure: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev), test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 1 jobs: ['7281771'] suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/xfstests-dev}'] Failure: Command failed (workunit test fs/fscrypt.sh) on smithi134 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=83caa3d7936db5043e7b3c387d3087f4af006726 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh none ffsb' 1 jobs: ['7281783'] suites: ['1-ceph', '2-logrotate}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'distro/{centos_latest}', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/v1', 'osd}', 'overrides/{distro/testing/k-testing}}', 'overrides/{ignorelist_health', 'pg-warn}', 'tasks/fscrypt-ffsb}'] Failure: Command failed (workunit test fs/test_python.sh) on smithi099 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=83caa3d7936db5043e7b3c387d3087f4af006726 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_python.sh' 1 jobs: ['7281802'] suites: ['1-ceph', '2-logrotate}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{ubuntu_20.04}', 'fs/libcephfs/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/libcephfs_python}'] Failure: Test failure: test_shutdown_killpoint_SHUTDOWN_EMPTYSUBTREES (tasks.cephfs.test_failover.TestShutdownKillpoints) 1 jobs: ['7281755'] suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/multifs/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon-debug}', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/failover}'] Failure: Cannot connect to remote host smithi026 4 jobs: ['7281796', '7281759', '7281803', '7281770'] suites intersection: ['conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}'] suites union: ['1-cephadm', '1-check-counter', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '2-logrotate}', '2-scrub/no', '2-scrub/yes', '3-inline/no', '3-inline/yes', '3-snaps/no', '3-snaps/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-flush/no', '4-flush/yes', '4-verify}', '5-upgrade-with-workload', '5-workunit/suites/blogbench}}', '5-workunit/suites/fsstress}}', '6-verify}}', 'begin/{0-install', 'centos_8.stream_container_tools', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fail_fs/yes', 'frag', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'ms_mode/legacy', 'n/3', 'n/5', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg-warn', 'ranks/multi/{export-check', 'replication/always}', 'replication/default}', 'rhel_8}', 'roles', 'session_timeout}', 'standby-replay', 'syntax}', 'tasks/{0-from/pacific', 'tasks/{0-subvolume/{with-namespace-isolated}', 'tasks/{0-subvolume/{with-quota}', 'wsync/no}', 'wsync/yes}'] Failure: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 1 jobs: ['7281801'] suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/quota}'] Failure: Test failure: test_open_inode (tasks.cephfs.test_strays.TestStrays) 1 jobs: ['7281765'] suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_20.04}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/strays}'] Failure: Test failure: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2 jobs: ['7281789', '7281762'] suites intersection: ['1-ceph', '2-logrotate}', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] suites union: ['1-ceph', '2-logrotate}', 'clusters/1-mds-2-client', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/cephfs_misc_tests}'] Failure: Command failed (workunit test kernel_untar_build.sh) on smithi047 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=83caa3d7936db5043e7b3c387d3087f4af006726 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2 jobs: ['7281785', '7281758'] suites intersection: ['2-logrotate}', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}'] suites union: ['1-ceph', '1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/kernel_untar_build}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'frag', 'fs/mixed-clients/{begin/{0-install', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'ms_mode/secure', 'n/5', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd-asserts}', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'overrides}}', 'ranks/multi/{export-check', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'tasks/{0-subvolume/{with-quota}', 'wsync/yes}'] Dead: None 8 jobs: ['7281776', '7281763', '7281766', '7281773', '7281784', '7281754', '7281757', '7281777'] Failure: error during scrub thrashing: rank damage found: {'backtrace'} 1 jobs: ['7281791'] suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/suites/blogbench}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'n/3', 'objectstore-ec/bluestore-comp', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'ranks/multi/{export-check', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-namespace-isolated-and-quota}', 'wsync/yes}'] Failure: 2 jobs: ['7281772', '7281794'] suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'prefetch_dirfrags/yes', 'races', 'ranks/3', 'session_timeout', 'thrashosds-health}'] suites union: ['1-ceph', '2-logrotate}', '2-workunit/suites/fsstress}}', '2-workunit/suites/pjd}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_20.04}', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'prefetch_dirfrags/yes', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/3', 'session_timeout', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'thrashosds-health}'] Failure: Command failed (workunit test suites/ffsb.sh) on smithi003 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=83caa3d7936db5043e7b3c387d3087f4af006726 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 1 jobs: ['7281793'] suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/suites/ffsb}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/secure', 'n/3', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'ranks/multi/{export-check', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-namespace-isolated-and-quota}', 'wsync/yes}'] Failure: Command failed on smithi164 with status 1: 'sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm' 1 jobs: ['7281760'] suites: ['1-ceph', '2-logrotate}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'distro/{centos_latest}', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/v1', 'osd}', 'overrides/{distro/testing/k-testing}}', 'overrides/{ignorelist_health', 'pg-warn}', 'tasks/fscrypt-common}'] Failure: {'smithi186.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'exception': 'Traceback (most recent call last):\r\n File "/home/ubuntu/.ansible/tmp/ansible-tmp-1684680080.1533403-5863-160785222538821/AnsiballZ_dnf.py", line 102, in \r\n _ansiballz_main()\r\n File "/home/ubuntu/.ansible/tmp/ansible-tmp-1684680080.1533403-5863-160785222538821/AnsiballZ_dnf.py", line 94, in _ansiballz_main\r\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n File "/home/ubuntu/.ansible/tmp/ansible-tmp-1684680080.1533403-5863-160785222538821/AnsiballZ_dnf.py", line 40, in invoke_module\r\n runpy.run_module(mod_name=\'ansible.modules.dnf\', init_globals=None, run_name=\'__main__\', alter_sys=True)\r\n File "/usr/lib64/python3.6/runpy.py", line 205, in run_module\r\n return _run_module_code(code, init_globals, run_name, mod_spec)\r\n File "/usr/lib64/python3.6/runpy.py", line 96, in _run_module_code\r\n mod_name, mod_spec, pkg_name, script_name)\r\n File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code\r\n exec(code, run_globals)\r\n File "/tmp/ansible_ansible.legacy.dnf_payload_0mn6ajd8/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1346, in \r\n File "/tmp/ansible_ansible.legacy.dnf_payload_0mn6ajd8/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1335, in main\r\n File "/tmp/ansible_ansible.legacy.dnf_payload_0mn6ajd8/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1310, in run\r\n File "/tmp/ansible_ansible.legacy.dnf_payload_0mn6ajd8/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1209, in ensure\r\n File "/usr/lib/python3.6/site-packages/dnf/base.py", line 1297, in _sig_check_pkg\r\n sigresult = dnf.rpm.miscutils.checkSig(ts, po.localPkg())\r\n File "/usr/lib/python3.6/site-packages/dnf/rpm/miscutils.py", line 102, in checkSig\r\n fdno = os.open(package, os.O_RDONLY|os.O_NOCTTY|os.O_CLOEXEC)\r\nFileNotFoundError: [Errno 2] No such file or directory: \'/var/cache/dnf/rhel-8-for-x86_64-appstream-rpms-4982e5444e88a9d1/packages/librbd1-12.2.7-9.el8.x86_64.rpm\'\r\n', 'module_stderr': 'Shared connection to smithi186.front.sepia.ceph.com closed.\r\n', 'module_stdout': 'Traceback (most recent call last):\r\n File "/home/ubuntu/.ansible/tmp/ansible-tmp-1684680080.1533403-5863-160785222538821/AnsiballZ_dnf.py", line 102, in \r\n _ansiballz_main()\r\n File "/home/ubuntu/.ansible/tmp/ansible-tmp-1684680080.1533403-5863-160785222538821/AnsiballZ_dnf.py", line 94, in _ansiballz_main\r\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n File "/home/ubuntu/.ansible/tmp/ansible-tmp-1684680080.1533403-5863-160785222538821/AnsiballZ_dnf.py", line 40, in invoke_module\r\n runpy.run_module(mod_name=\'ansible.modules.dnf\', init_globals=None, run_name=\'__main__\', alter_sys=True)\r\n File "/usr/lib64/python3.6/runpy.py", line 205, in run_module\r\n return _run_module_code(code, init_globals, run_name, mod_spec)\r\n File "/usr/lib64/python3.6/runpy.py", line 96, in _run_module_code\r\n mod_name, mod_spec, pkg_name, script_name)\r\n File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code\r\n exec(code, run_globals)\r\n File "/tmp/ansible_ansible.legacy.dnf_payload_0mn6ajd8/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1346, in \r\n File "/tmp/ansible_ansible.legacy.dnf_payload_0mn6ajd8/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1335, in main\r\n File "/tmp/ansible_ansible.legacy.dnf_payload_0mn6ajd8/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1310, in run\r\n File "/tmp/ansible_ansible.legacy.dnf_payload_0mn6ajd8/ansible_ansible.legacy.dnf_payload.zip/ansible/modules/dnf.py", line 1209, in ensure\r\n File "/usr/lib/python3.6/site-packages/dnf/base.py", line 1297, in _sig_check_pkg\r\n sigresult = dnf.rpm.miscutils.checkSig(ts, po.localPkg())\r\n File "/usr/lib/python3.6/site-packages/dnf/rpm/miscutils.py", line 102, in checkSig\r\n fdno = os.open(package, os.O_RDONLY|os.O_NOCTTY|os.O_CLOEXEC)\r\nFileNotFoundError: [Errno 2] No such file or directory: \'/var/cache/dnf/rhel-8-for-x86_64-appstream-rpms-4982e5444e88a9d1/packages/librbd1-12.2.7-9.el8.x86_64.rpm\'\r\n', 'msg': 'MODULE FAILURE\nSee stdout/stderr for the exact error', 'rc': 1}} 1 jobs: ['7281767'] suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/test_journal_migration}'] Dead: b'2023-05-22T02:25:01.914 DEBUG:teuthology.exit:Finished running handlers' 1 jobs: ['7281756'] suites: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '3-inline/no', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/no', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/v16.2.4'] Timeout 3h running clone.client.0/qa/workunits/suites/dbench.sh 3 jobs: ['7281795', '7281774', '7281792'] suites intersection: ['2-logrotate}', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'osd}', 'ranks/1', 'session_timeout}'] suites union: ['1-ceph', '1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/no', '2-scrub/yes', '3-snaps/no', '3-snaps/yes', '4-flush/no', '4-flush/yes', '5-workunit/suites/dbench}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'frag', 'fs/verify/{begin/{0-install', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'ms_mode/secure', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'ranks/1', 'rhel_8}', 'session_timeout}', 'standby-replay', 'tasks/dbench', 'tasks/{0-subvolume/{no-subvolume}', 'tasks/{0-subvolume/{with-no-extra-options}', 'validater/valgrind}', 'wsync/no}', 'wsync/yes}'] Crash: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi002 with status 135: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=83caa3d7936db5043e7b3c387d3087f4af006726 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' ceph version 18.0.0-4092-gf704846c (f704846cae03d45131067a3ffbcec6231d97621a) reef (dev) 1: /lib64/libpthread.so.0(+0x12ce0) [0x7fc2623d0ce0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc2631cbe19] 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7fc2631cbf85] 6: (Client::add_update_cap(Inode*, MetaSession*, unsigned long, unsigned int, unsigned int, unsigned int, unsigned int, inodeno_t, int, UserPerm const&)+0xc53) [0x564388b0a2f3] 7: (Client::add_update_inode(InodeStat*, utime_t, MetaSession*, UserPerm const&)+0x943) [0x564388b0b6c3] 8: (Client::insert_trace(MetaRequest*, MetaSession*)+0xd6e) [0x564388b0eb3e] 9: (Client::handle_client_reply(boost::intrusive_ptr const&)+0x3a0) [0x564388b0f900] 10: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x58d) [0x564388b32c0d] 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7fc263444f58] 12: (DispatchQueue::entry()+0x50f) [0x7fc2634420ff] 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc263508451] 14: /lib64/libpthread.so.0(+0x81cf) [0x7fc2623c61cf] 15: clone() 2 jobs: ['7281764', '7281786'] suites intersection: ['2-logrotate}', 'conf/{client', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'osd}'] suites union: ['1-ceph', '1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/no', '2-workunit/fs/snaps}}', '3-snaps/no', '4-flush/no', '5-workunit/postgres}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'frag', 'fs/thrash/workloads/{begin/{0-install', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'n/5', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{frag', 'prefetch_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'races', 'ranks/5', 'ranks/multi/{export-check', 'replication/always}', 'session_timeout', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-quota}', 'tasks/{1-thrash/mds', 'thrashosds-health}'] Failure: Test failure: test_evicted_caps (tasks.cephfs.test_client_recovery.TestClientRecovery) 1 jobs: ['7281782'] suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/client-recovery}'] Failure: {'smithi165.front.sepia.ceph.com': {'changed': False, 'msg': 'All items completed', 'results': [{'_ansible_item_label': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, '_ansible_no_log': False, 'ansible_loop_var': 'item', 'changed': False, 'err': " /dev/vg_nvme: already exists in filesystem\n Run `vgcreate --help' for more information.\n", 'failed': True, 'invocation': {'module_args': {'force': False, 'pesize': '4', 'pv_options': '', 'pvresize': False, 'pvs': ['/dev/nvme0n1'], 'state': 'present', 'vg': 'vg_nvme', 'vg_options': ''}}, 'item': {'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}, 'msg': "Creating volume group 'vg_nvme' failed", 'rc': 3}]}} 1 jobs: ['7281768'] suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/postgres}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'n/5', 'objectstore-ec/bluestore-comp', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'ranks/multi/{export-check', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-quota}', 'wsync/yes}'] Failure: reached maximum tries (50) after waiting for 300 seconds 1 jobs: ['7281761'] suites: ['1-ceph', '2-logrotate}', 'cephfs-mirror/three-per-cluster', 'clients/{mirror}', 'cluster/{1-node}', 'fs/mirror-ha/{begin/{0-install', 'objectstore/bluestore-bitmap', 'overrides/{whitelist_health}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/cephfs-mirror-ha-workunit}'] Failure: Command failed (workunit test libcephfs/test.sh) on smithi061 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=83caa3d7936db5043e7b3c387d3087f4af006726 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh' 1 jobs: ['7281788'] suites: ['1-ceph', '2-logrotate}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{ubuntu_20.04}', 'fs/libcephfs/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/libcephfs/{frag', 'test}}']