2022-11-27T16:52:01.575 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-27T16:52:01.576 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --verbose --name yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088680 --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} -- /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088680/orig.config.yaml 2022-11-27T16:52:01.633 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-11-27T16:52:01.715 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088680 branch: main description: fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} email: ceph-qa@ceph.io first_in_suite: false job_id: '7088680' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'install ceph/octopus latest ' name: yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: main ceph: cephfs: max_mds: 2 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on osd down out interval zero: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - mds.a - mds.b - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - client.0 - - client.1 sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 sleep_before_teardown: 0 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 targets: smithi154.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRwDQ8CUvWoPdXlOJeXVcW6i+kWl13AL6AR3u3Ytaom0t+b033huDjTeDl3anfRE6SDz1vRVIFmsQa47CsRu8MHFhN+lCI8+MEaytuud/Fl8jwPkZYydr86jBL+3rFf4tiMEHdQxStmlLYFKEYtLGhYYhae7lKKTF1fcxMai+7ysa2Lpd6sXNusQvCZT4X4IYhFVF1VfAEk9zKJUaSNc43ltE1v7SoQRTfuBFj4zCLn/m//RvJ4syDwAUk6Z58bkLEAB73h34S8keSre2utsJ6byT3hq7sx7nXtA6XSjZVj/ruVEuLKnOPWNevD4w7siX5mX1R/jNF5zt8z7CZXKUBIiw1wkqj94f/rbROftEXWUXXLffzDMYddle4+8LIH6Ya+PbobAxjm682/qEhspjtIiyavCwMWsIrHkgiVNsDAoyPg5G/yTWjZsPysgbKNKK6muWjv6qvo5SVWC5jXYh8b3tX9ufW40iiuDh7S4Pg5b2v0S4Ih2/PlVUTWk34nqU= smithi159.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJFWkONOX1m/IQoBLaXRPuRnuyN5L2gbWLlcTMGGh1dUsGISOV8Wfx4n0adCkI9Wh5FyrrqzkH2Jm+Fo70O9EBKuCMrtOTH+1CV1x4IAo+BEwC1VnKI5LTqzSADPmD67AZ8wEygDMFTlRQwR9Ok+/8V49Ky/CJ21ju9EwBoW/YD7pmKIIfnLC4falwhgkuAihXgCO4cUC+Z6/ndUt7s4St5V7Zz68gdlPWNWwldCC9Xsr0RurTSXt09tVxAcY2c59tPiLg7fuKKV7NpZZ/g5qxgP+t14jnb4YG4D5AmqpV+/oZ5BVHrhfkYn2C4OuaDZz+WOHLA8pPAvZAKFCpLZYpcHDF1POUI+9BbJs+VRLliI40emWs7fXCwqgLsPSjXbIhYeQgmtE1hGJ84MA6EA5SIXLiih/SYfENGDveLBONgxH5TCRdtnDla7ABueu34t3UDxQ+yotvwQqglxqnOl3XCkDeh68KVObvyy+Z7++NYJ0qoSDVmcWEq3Cz8XZkprc= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAy9i2EOsT8UOx7LgqTIHa8mfUt2zFMUe4jNDlRaCBnJHablrIgUN0sbyGOfRvtCSFidJ8HZPQL9qtfZW9fdmSw= tasks: - install: branch: octopus exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-rook - ceph-mgr-cephadm - cephadm - ceph-volume extra_packages: - librados2 - print: '**** done installing octopus' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client octopus - print: '**** done ceph' - ceph-fuse: null - print: '**** done octopus client' - workunit: clients: all: - suites/fsstress.sh - print: '**** done fsstress' - mds_pre_upgrade: null - print: '**** done mds pre-upgrade sequence' - install.upgrade: mon.a: branch: quincy mon.b: branch: quincy - print: '**** done install.upgrade both hosts' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - ceph.stop: - mds.* - ceph.restart: daemons: - mds.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph osd dump -f json-pretty - ceph versions - ceph osd require-osd-release quincy - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph.healthy: null - print: '**** done ceph.restart' - workunit: clients: all: - suites/fsstress.sh - print: '**** done fsstress' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-23_17:24:26 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.20399 2022-11-27T16:52:01.715 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa; will attempt to use it 2022-11-27T16:52:01.716 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa/tasks 2022-11-27T16:52:01.716 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-11-27T16:52:01.719 INFO:teuthology.task.internal:Checking packages... 2022-11-27T16:52:01.747 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '697b0d99d9a52df92b20c8e053dcd46c75f364a9' 2022-11-27T16:52:01.748 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T16:52:01.748 INFO:teuthology.packaging:ref: None 2022-11-27T16:52:01.749 INFO:teuthology.packaging:tag: None 2022-11-27T16:52:01.749 INFO:teuthology.packaging:branch: main 2022-11-27T16:52:01.749 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:52:01.750 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=main 2022-11-27T16:52:01.994 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-995.gaf6ed506 2022-11-27T16:52:01.994 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-11-27T16:52:02.002 INFO:teuthology.task.internal:no buildpackages task found 2022-11-27T16:52:02.002 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-11-27T16:52:02.037 INFO:teuthology.task.internal:Saving configuration 2022-11-27T16:52:02.060 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-11-27T16:52:02.069 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-27T16:52:02.091 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088680', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 16:46:46.526849', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPoQH8A8aGxMXbedegTXFTR7PyI5iTwtOx5caAO2XL1uw5J6YypvKyuh2RMzLlgiLa8X7RdaIslXL5TzXshD4VQ='} 2022-11-27T16:52:02.113 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088680', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 16:46:46.525791', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAwCgaCcw3nxSX+HxYkQp8dESuZTD07VyYvDOWFVboJeBT43sih6C1h8pTSbQttdnDKsTRatEoMa1S1cfefAoTg='} 2022-11-27T16:52:02.140 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088680', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 16:46:46.524184', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAy9i2EOsT8UOx7LgqTIHa8mfUt2zFMUe4jNDlRaCBnJHablrIgUN0sbyGOfRvtCSFidJ8HZPQL9qtfZW9fdmSw='} 2022-11-27T16:52:02.140 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-11-27T16:52:02.148 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-11-27T16:52:02.149 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['client.0'] 2022-11-27T16:52:02.149 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['client.1'] 2022-11-27T16:52:02.149 INFO:teuthology.run_tasks:Running task console_log... 2022-11-27T16:52:02.281 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f27acc141e0>, signals=[15]) 2022-11-27T16:52:02.283 INFO:teuthology.run_tasks:Running task internal.connect... 2022-11-27T16:52:02.294 INFO:teuthology.task.internal:Opening connections... 2022-11-27T16:52:02.295 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2022-11-27T16:52:02.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:52:02.387 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2022-11-27T16:52:02.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:52:02.462 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2022-11-27T16:52:02.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:52:02.547 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-11-27T16:52:02.559 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2022-11-27T16:52:02.581 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2022-11-27T16:52:02.581 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2022-11-27T16:52:02.641 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2022-11-27T16:52:02.641 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8" 2022-11-27T16:52:02.642 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2022-11-27T16:52:02.642 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2022-11-27T16:52:02.642 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8" 2022-11-27T16:52:02.642 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8" 2022-11-27T16:52:02.643 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 8" 2022-11-27T16:52:02.643 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2022-11-27T16:52:02.643 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-11-27T16:52:02.643 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2022-11-27T16:52:02.644 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T16:52:02.644 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T16:52:02.644 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-11-27T16:52:02.645 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2022-11-27T16:52:02.665 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2022-11-27T16:52:02.683 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2022-11-27T16:52:02.684 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2022-11-27T16:52:02.740 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2022-11-27T16:52:02.741 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="8" 2022-11-27T16:52:02.741 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2022-11-27T16:52:02.741 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2022-11-27T16:52:02.741 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="8" 2022-11-27T16:52:02.742 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el8" 2022-11-27T16:52:02.742 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 8" 2022-11-27T16:52:02.742 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2022-11-27T16:52:02.742 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-11-27T16:52:02.743 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2022-11-27T16:52:02.743 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T16:52:02.743 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T16:52:02.744 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-11-27T16:52:02.744 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-11-27T16:52:02.763 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2022-11-27T16:52:02.785 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2022-11-27T16:52:02.785 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2022-11-27T16:52:02.844 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2022-11-27T16:52:02.845 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="8" 2022-11-27T16:52:02.845 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2022-11-27T16:52:02.845 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2022-11-27T16:52:02.845 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="8" 2022-11-27T16:52:02.846 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el8" 2022-11-27T16:52:02.846 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 8" 2022-11-27T16:52:02.846 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2022-11-27T16:52:02.847 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-11-27T16:52:02.847 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2022-11-27T16:52:02.847 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T16:52:02.848 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T16:52:02.848 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-11-27T16:52:02.848 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2022-11-27T16:52:02.866 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-11-27T16:52:02.880 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-11-27T16:52:02.914 INFO:teuthology.task.internal:Checking for old test directory... 2022-11-27T16:52:02.914 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2022-11-27T16:52:02.918 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2022-11-27T16:52:02.921 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2022-11-27T16:52:02.939 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-11-27T16:52:02.950 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-11-27T16:52:02.950 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2022-11-27T16:52:02.977 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2022-11-27T16:52:02.981 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2022-11-27T16:52:03.007 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-11-27T16:52:03.086 INFO:teuthology.run_tasks:Running task kernel... 2022-11-27T16:52:03.106 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-11-27T16:52:03.107 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-11-27T16:52:03.108 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T16:52:03.109 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T16:52:03.109 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T16:52:03.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:52:03.127 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2022-11-27T16:52:03.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:52:03.129 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2022-11-27T16:52:03.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:52:03.130 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2022-11-27T16:52:03.184 INFO:teuthology.orchestra.run.smithi159.stdout:4.18.0-408.el8.x86_64 2022-11-27T16:52:03.185 INFO:teuthology.task.kernel:Running kernel on smithi159: 4.18.0-408.el8.x86_64 2022-11-27T16:52:03.185 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum install -y kernel 2022-11-27T16:52:03.187 INFO:teuthology.orchestra.run.smithi165.stdout:4.18.0-408.el8.x86_64 2022-11-27T16:52:03.188 INFO:teuthology.task.kernel:Running kernel on smithi165: 4.18.0-408.el8.x86_64 2022-11-27T16:52:03.188 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum install -y kernel 2022-11-27T16:52:03.190 INFO:teuthology.orchestra.run.smithi154.stdout:4.18.0-408.el8.x86_64 2022-11-27T16:52:03.190 INFO:teuthology.task.kernel:Running kernel on smithi154: 4.18.0-408.el8.x86_64 2022-11-27T16:52:03.190 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum install -y kernel 2022-11-27T16:52:06.530 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-8 - AppStream 29 MB/s | 26 MB 00:00 2022-11-27T16:52:06.606 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS-8 - AppStream 25 MB/s | 26 MB 00:01 2022-11-27T16:52:06.767 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-8 - AppStream 25 MB/s | 26 MB 00:01 2022-11-27T16:52:12.008 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-8 - BaseOS 31 MB/s | 26 MB 00:00 2022-11-27T16:52:12.086 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS-8 - BaseOS 30 MB/s | 26 MB 00:00 2022-11-27T16:52:12.134 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-8 - BaseOS 34 MB/s | 26 MB 00:00 2022-11-27T16:52:16.004 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-8 - Extras 160 kB/s | 18 kB 00:00 2022-11-27T16:52:16.064 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS-8 - Extras 132 kB/s | 18 kB 00:00 2022-11-27T16:52:16.141 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-8 - Extras 182 kB/s | 18 kB 00:00 2022-11-27T16:52:16.372 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-8 - PowerTools 21 MB/s | 5.3 MB 00:00 2022-11-27T16:52:16.449 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS-8 - PowerTools 20 MB/s | 5.3 MB 00:00 2022-11-27T16:52:16.502 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-8 - PowerTools 22 MB/s | 5.3 MB 00:00 2022-11-27T16:52:19.757 INFO:teuthology.orchestra.run.smithi165.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2022-11-27T16:52:20.857 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:00 2022-11-27T16:52:21.680 INFO:teuthology.orchestra.run.smithi159.stdout:Copr repo for python3-asyncssh owned by ceph 827 B/s | 3.5 kB 00:04 2022-11-27T16:52:21.992 INFO:teuthology.orchestra.run.smithi154.stdout:Copr repo for python3-asyncssh owned by ceph 793 B/s | 3.5 kB 00:04 2022-11-27T16:52:22.848 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:01 2022-11-27T16:52:23.092 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:00 2022-11-27T16:52:24.040 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 437 kB/s | 24 kB 00:00 2022-11-27T16:52:26.030 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 426 kB/s | 24 kB 00:00 2022-11-27T16:52:26.194 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:01 ago on Sun 27 Nov 2022 04:52:25 PM UTC. 2022-11-27T16:52:26.317 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 407 kB/s | 24 kB 00:00 2022-11-27T16:52:27.021 INFO:teuthology.orchestra.run.smithi165.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2022-11-27T16:52:27.022 INFO:teuthology.orchestra.run.smithi165.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2022-11-27T16:52:27.156 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T16:52:27.157 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T16:52:27.157 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T16:52:27.271 DEBUG:teuthology.orchestra.run.smithi165:> echo no | sudo yum reinstall kernel || true 2022-11-27T16:52:27.860 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:04 ago on Sun 27 Nov 2022 04:52:23 PM UTC. 2022-11-27T16:52:28.936 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T16:52:28.938 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T16:52:28.938 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2022-11-27T16:52:28.938 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T16:52:28.938 INFO:teuthology.orchestra.run.smithi165.stdout:Reinstalling: 2022-11-27T16:52:28.939 INFO:teuthology.orchestra.run.smithi165.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T16:52:28.939 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T16:52:28.939 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T16:52:28.939 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T16:52:28.940 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T16:52:28.940 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 8.5 M 2022-11-27T16:52:28.940 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 0 2022-11-27T16:52:28.941 INFO:teuthology.orchestra.run.smithi165.stderr:Operation aborted. 2022-11-27T16:52:29.012 INFO:teuthology.orchestra.run.smithi165.stdout:Is this ok [y/N]: 2022-11-27T16:52:29.012 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum reinstall -y kernel || true 2022-11-27T16:52:29.061 INFO:teuthology.orchestra.run.smithi159.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2022-11-27T16:52:29.061 INFO:teuthology.orchestra.run.smithi159.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2022-11-27T16:52:29.190 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T16:52:29.191 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T16:52:29.191 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T16:52:29.232 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2022-11-27T16:52:29.233 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2022-11-27T16:52:29.323 DEBUG:teuthology.orchestra.run.smithi159:> echo no | sudo yum reinstall kernel || true 2022-11-27T16:52:29.390 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T16:52:29.391 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T16:52:29.391 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T16:52:29.499 DEBUG:teuthology.orchestra.run.smithi154:> echo no | sudo yum reinstall kernel || true 2022-11-27T16:52:29.602 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:06 ago on Sun 27 Nov 2022 04:52:23 PM UTC. 2022-11-27T16:52:29.907 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:04 ago on Sun 27 Nov 2022 04:52:25 PM UTC. 2022-11-27T16:52:30.085 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:04 ago on Sun 27 Nov 2022 04:52:26 PM UTC. 2022-11-27T16:52:30.680 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T16:52:30.682 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T16:52:30.682 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2022-11-27T16:52:30.682 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T16:52:30.682 INFO:teuthology.orchestra.run.smithi165.stdout:Reinstalling: 2022-11-27T16:52:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T16:52:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T16:52:30.683 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T16:52:30.683 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T16:52:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T16:52:30.684 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 8.5 M 2022-11-27T16:52:30.684 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 0 2022-11-27T16:52:30.684 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T16:52:30.921 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-4.18.0-408.el8.x86_64.rpm 36 MB/s | 8.5 MB 00:00 2022-11-27T16:52:30.922 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:52:30.922 INFO:teuthology.orchestra.run.smithi165.stdout:Total 36 MB/s | 8.5 MB 00:00 2022-11-27T16:52:30.994 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T16:52:30.996 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T16:52:30.996 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2022-11-27T16:52:30.996 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T16:52:30.996 INFO:teuthology.orchestra.run.smithi159.stdout:Reinstalling: 2022-11-27T16:52:30.997 INFO:teuthology.orchestra.run.smithi159.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T16:52:30.997 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T16:52:30.997 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T16:52:30.997 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T16:52:30.998 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T16:52:30.998 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 8.5 M 2022-11-27T16:52:30.998 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 0 2022-11-27T16:52:30.998 INFO:teuthology.orchestra.run.smithi159.stderr:Operation aborted. 2022-11-27T16:52:31.016 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T16:52:31.062 INFO:teuthology.orchestra.run.smithi159.stdout:Is this ok [y/N]: 2022-11-27T16:52:31.063 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum reinstall -y kernel || true 2022-11-27T16:52:31.176 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T16:52:31.178 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T16:52:31.178 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2022-11-27T16:52:31.179 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T16:52:31.179 INFO:teuthology.orchestra.run.smithi154.stdout:Reinstalling: 2022-11-27T16:52:31.179 INFO:teuthology.orchestra.run.smithi154.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T16:52:31.180 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T16:52:31.180 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T16:52:31.180 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T16:52:31.181 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T16:52:31.181 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 8.5 M 2022-11-27T16:52:31.181 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 0 2022-11-27T16:52:31.182 INFO:teuthology.orchestra.run.smithi154.stderr:Operation aborted. 2022-11-27T16:52:31.244 INFO:teuthology.orchestra.run.smithi154.stdout:Is this ok [y/N]: 2022-11-27T16:52:31.244 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum reinstall -y kernel || true 2022-11-27T16:52:31.435 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T16:52:31.436 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T16:52:31.607 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T16:52:31.622 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T16:52:31.640 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:06 ago on Sun 27 Nov 2022 04:52:25 PM UTC. 2022-11-27T16:52:31.823 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:05 ago on Sun 27 Nov 2022 04:52:26 PM UTC. 2022-11-27T16:52:31.892 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T16:52:32.106 INFO:teuthology.orchestra.run.smithi165.stdout: Reinstalling : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T16:52:32.107 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T16:52:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T16:52:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T16:52:32.720 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T16:52:32.721 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T16:52:32.721 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2022-11-27T16:52:32.721 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T16:52:32.722 INFO:teuthology.orchestra.run.smithi159.stdout:Reinstalling: 2022-11-27T16:52:32.722 INFO:teuthology.orchestra.run.smithi159.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T16:52:32.722 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T16:52:32.722 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T16:52:32.722 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T16:52:32.723 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T16:52:32.723 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 8.5 M 2022-11-27T16:52:32.723 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 0 2022-11-27T16:52:32.723 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T16:52:32.901 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T16:52:32.903 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T16:52:32.903 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2022-11-27T16:52:32.903 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T16:52:32.903 INFO:teuthology.orchestra.run.smithi154.stdout:Reinstalling: 2022-11-27T16:52:32.904 INFO:teuthology.orchestra.run.smithi154.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T16:52:32.904 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T16:52:32.904 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T16:52:32.904 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T16:52:32.905 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T16:52:32.905 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 8.5 M 2022-11-27T16:52:32.905 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 0 2022-11-27T16:52:32.905 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T16:52:32.923 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-4.18.0-408.el8.x86_64.rpm 43 MB/s | 8.5 MB 00:00 2022-11-27T16:52:32.924 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:52:32.924 INFO:teuthology.orchestra.run.smithi159.stdout:Total 42 MB/s | 8.5 MB 00:00 2022-11-27T16:52:33.021 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T16:52:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T16:52:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T16:52:33.049 INFO:teuthology.orchestra.run.smithi165.stdout:Reinstalled: 2022-11-27T16:52:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: kernel-4.18.0-408.el8.x86_64 2022-11-27T16:52:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T16:52:33.050 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T16:52:33.119 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-4.18.0-408.el8.x86_64.rpm 40 MB/s | 8.5 MB 00:00 2022-11-27T16:52:33.120 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:52:33.121 INFO:teuthology.orchestra.run.smithi154.stdout:Total 39 MB/s | 8.5 MB 00:00 2022-11-27T16:52:33.161 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q kernel --last | head -n 1 2022-11-27T16:52:33.217 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T16:52:33.293 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-4.18.0-408.el8.x86_64 Sun 27 Nov 2022 04:52:31 PM UTC 2022-11-27T16:52:33.295 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-408.el8.x86_64 2022-11-27T16:52:33.296 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T16:52:33.296 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2022-11-27T16:52:33.296 DEBUG:teuthology.orchestra.run.smithi165:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-27T16:52:33.335 INFO:teuthology.orchestra.run.smithi165.stdout:ttyS1 2022-11-27T16:52:33.351 DEBUG:teuthology.parallel:result is None 2022-11-27T16:52:33.441 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T16:52:33.442 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T16:52:33.614 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T16:52:33.614 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T16:52:33.619 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T16:52:33.632 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T16:52:33.801 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T16:52:33.827 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T16:52:33.913 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T16:52:34.121 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T16:52:34.135 INFO:teuthology.orchestra.run.smithi159.stdout: Reinstalling : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T16:52:34.136 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T16:52:34.360 INFO:teuthology.orchestra.run.smithi154.stdout: Reinstalling : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T16:52:34.360 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T16:52:34.583 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T16:52:34.584 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T16:52:34.750 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T16:52:34.750 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T16:52:35.064 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T16:52:35.065 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T16:52:35.065 INFO:teuthology.orchestra.run.smithi159.stdout:Reinstalled: 2022-11-27T16:52:35.065 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-4.18.0-408.el8.x86_64 2022-11-27T16:52:35.066 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T16:52:35.066 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T16:52:35.179 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q kernel --last | head -n 1 2022-11-27T16:52:35.312 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-4.18.0-408.el8.x86_64 Sun 27 Nov 2022 04:52:33 PM UTC 2022-11-27T16:52:35.314 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-408.el8.x86_64 2022-11-27T16:52:35.315 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T16:52:35.315 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2022-11-27T16:52:35.315 DEBUG:teuthology.orchestra.run.smithi159:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-27T16:52:35.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T16:52:35.323 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T16:52:35.324 INFO:teuthology.orchestra.run.smithi154.stdout:Reinstalled: 2022-11-27T16:52:35.324 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-4.18.0-408.el8.x86_64 2022-11-27T16:52:35.325 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T16:52:35.325 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T16:52:35.351 INFO:teuthology.orchestra.run.smithi159.stdout:ttyS1 2022-11-27T16:52:35.366 DEBUG:teuthology.parallel:result is None 2022-11-27T16:52:35.455 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q kernel --last | head -n 1 2022-11-27T16:52:35.587 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-4.18.0-408.el8.x86_64 Sun 27 Nov 2022 04:52:34 PM UTC 2022-11-27T16:52:35.588 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-408.el8.x86_64 2022-11-27T16:52:35.588 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T16:52:35.589 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-11-27T16:52:35.589 DEBUG:teuthology.orchestra.run.smithi154:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-27T16:52:35.627 INFO:teuthology.orchestra.run.smithi154.stdout:ttyS1 2022-11-27T16:52:35.643 DEBUG:teuthology.parallel:result is None 2022-11-27T16:52:35.643 INFO:teuthology.run_tasks:Running task internal.base... 2022-11-27T16:52:35.652 INFO:teuthology.task.internal:Creating test directory... 2022-11-27T16:52:35.653 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T16:52:35.686 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T16:52:35.688 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T16:52:35.710 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-11-27T16:52:35.718 INFO:teuthology.run_tasks:Running task internal.archive... 2022-11-27T16:52:35.746 INFO:teuthology.task.internal:Creating archive directory... 2022-11-27T16:52:35.747 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T16:52:35.749 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T16:52:35.752 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T16:52:35.802 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-11-27T16:52:35.820 INFO:teuthology.task.internal:Enabling coredump saving... 2022-11-27T16:52:35.821 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T16:52:35.824 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T16:52:35.829 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T16:52:35.867 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:52:35.868 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:52:35.876 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:52:35.879 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:52:35.879 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:52:35.890 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:52:35.892 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-11-27T16:52:35.905 INFO:teuthology.task.internal:Configuring sudo... 2022-11-27T16:52:35.906 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T16:52:35.909 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T16:52:35.920 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T16:52:35.962 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-11-27T16:52:35.972 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-11-27T16:52:35.972 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T16:52:35.977 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T16:52:35.987 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T16:52:36.021 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:52:36.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:52:36.127 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:52:36.213 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:52:36.254 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T16:52:36.255 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T16:52:36.321 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:52:36.367 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:52:36.418 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:52:36.503 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:52:36.545 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2022-11-27T16:52:36.546 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T16:52:36.612 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:52:36.662 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:52:36.720 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:52:36.770 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:52:36.814 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2022-11-27T16:52:36.815 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T16:52:36.884 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2022-11-27T16:52:36.888 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2022-11-27T16:52:36.891 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2022-11-27T16:52:36.942 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T16:52:36.951 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T16:52:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T16:52:37.391 INFO:teuthology.run_tasks:Running task internal.timer... 2022-11-27T16:52:37.404 INFO:teuthology.task.internal:Starting timer... 2022-11-27T16:52:37.404 INFO:teuthology.run_tasks:Running task pcp... 2022-11-27T16:52:37.433 INFO:teuthology.run_tasks:Running task selinux... 2022-11-27T16:52:37.465 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2022-11-27T16:52:37.466 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2022-11-27T16:52:37.538 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2022-11-27T16:52:37.540 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2022-11-27T16:52:37.591 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2022-11-27T16:52:37.593 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2022-11-27T16:52:37.664 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2022-11-27T16:52:37.665 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-11-27T16:52:37.666 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2022-11-27T16:52:37.702 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2022-11-27T16:52:37.703 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2022-11-27T16:52:37.752 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2022-11-27T16:52:37.752 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2022-11-27T16:52:37.786 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2022-11-27T16:52:37.787 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi154.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive'} 2022-11-27T16:52:37.787 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T16:52:37.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:52:37.815 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T16:52:37.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:52:37.844 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T16:52:37.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:52:37.872 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-11-27T16:52:37.872 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2022-11-27T16:52:37.906 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2022-11-27T16:52:37.961 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2022-11-27T16:52:38.004 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-11-27T16:52:38.018 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2022-11-27T16:52:38.019 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-11-27T16:52:38.038 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-11-27T16:52:38.040 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi154.front.sepia.ceph.com,smithi159.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-11-27T16:59:33.838 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi154.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2022-11-27T16:59:33.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2022-11-27T16:59:33.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:59:33.934 DEBUG:teuthology.orchestra.run.smithi154:> true 2022-11-27T16:59:34.005 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2022-11-27T16:59:34.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:59:34.081 DEBUG:teuthology.orchestra.run.smithi159:> true 2022-11-27T16:59:34.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2022-11-27T16:59:34.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:59:34.233 DEBUG:teuthology.orchestra.run.smithi165:> true 2022-11-27T16:59:34.306 INFO:teuthology.run_tasks:Running task clock... 2022-11-27T16:59:34.319 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-11-27T16:59:34.320 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:59:34.323 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:59:34.325 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:59:34.326 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:59:34.328 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:59:34.329 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:59:34.354 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:59:34.357 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:59:34.368 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:59:34.372 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:59:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:59:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:59:34.442 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2022-11-27T16:59:34.442 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2022-11-27T16:59:34.453 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2022-11-27T16:59:34.454 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2022-11-27T16:59:34.458 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2022-11-27T16:59:34.468 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:59:34.468 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:59:34.470 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2022-11-27T16:59:34.481 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:59:34.482 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:59:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:59:34.497 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:59:34.540 INFO:teuthology.orchestra.run.smithi159.stderr:bash: ntpq: command not found 2022-11-27T16:59:34.543 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:59:34.544 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2022-11-27T16:59:34.544 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.544 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.544 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.544 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.547 INFO:teuthology.orchestra.run.smithi154.stderr:bash: ntpq: command not found 2022-11-27T16:59:34.550 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:59:34.551 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2022-11-27T16:59:34.551 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.551 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.551 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.552 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.556 INFO:teuthology.orchestra.run.smithi165.stderr:bash: ntpq: command not found 2022-11-27T16:59:34.560 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:59:34.560 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2022-11-27T16:59:34.561 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.561 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.561 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.562 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:59:34.562 INFO:teuthology.run_tasks:Running task install... 2022-11-27T16:59:34.571 DEBUG:teuthology.task.install:project ceph 2022-11-27T16:59:34.572 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}} 2022-11-27T16:59:34.572 DEBUG:teuthology.task.install:config {'branch': 'octopus', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'} 2022-11-27T16:59:34.573 INFO:teuthology.task.install:Using flavor: default 2022-11-27T16:59:34.579 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-11-27T16:59:34.580 INFO:teuthology.task.install:extra packages: ['librados2'] 2022-11-27T16:59:34.580 DEBUG:teuthology.orchestra.run.smithi154:> sudo lsb_release -is 2022-11-27T16:59:34.631 INFO:teuthology.orchestra.run.smithi154.stdout:CentOSStream 2022-11-27T16:59:34.632 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:59:34.633 DEBUG:teuthology.orchestra.run.smithi159:> sudo lsb_release -is 2022-11-27T16:59:34.633 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'octopus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:59:34.634 DEBUG:teuthology.orchestra.run.smithi154:> sudo lsb_release -is 2022-11-27T16:59:34.671 INFO:teuthology.orchestra.run.smithi159.stdout:CentOSStream 2022-11-27T16:59:34.672 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:59:34.673 DEBUG:teuthology.orchestra.run.smithi165:> sudo lsb_release -is 2022-11-27T16:59:34.673 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'octopus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:59:34.674 DEBUG:teuthology.orchestra.run.smithi159:> sudo lsb_release -is 2022-11-27T16:59:34.709 INFO:teuthology.orchestra.run.smithi154.stdout:CentOSStream 2022-11-27T16:59:34.711 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:59:34.711 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T16:59:34.712 INFO:teuthology.packaging:ref: None 2022-11-27T16:59:34.712 INFO:teuthology.packaging:tag: None 2022-11-27T16:59:34.712 INFO:teuthology.packaging:branch: octopus 2022-11-27T16:59:34.713 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:59:34.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T16:59:34.720 INFO:teuthology.orchestra.run.smithi165.stdout:CentOSStream 2022-11-27T16:59:34.720 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:59:34.721 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'octopus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:59:34.721 DEBUG:teuthology.orchestra.run.smithi165:> sudo lsb_release -is 2022-11-27T16:59:34.754 INFO:teuthology.orchestra.run.smithi159.stdout:CentOSStream 2022-11-27T16:59:34.755 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:59:34.755 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T16:59:34.756 INFO:teuthology.packaging:ref: None 2022-11-27T16:59:34.756 INFO:teuthology.packaging:tag: None 2022-11-27T16:59:34.756 INFO:teuthology.packaging:branch: octopus 2022-11-27T16:59:34.757 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:59:34.757 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T16:59:34.796 INFO:teuthology.orchestra.run.smithi165.stdout:CentOSStream 2022-11-27T16:59:34.797 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:59:34.798 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T16:59:34.798 INFO:teuthology.packaging:ref: None 2022-11-27T16:59:34.798 INFO:teuthology.packaging:tag: None 2022-11-27T16:59:34.799 INFO:teuthology.packaging:branch: octopus 2022-11-27T16:59:34.799 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:59:34.799 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T16:59:34.930 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/ 2022-11-27T16:59:34.931 INFO:teuthology.task.install.rpm:Package version is 15.2.17-0 2022-11-27T16:59:34.988 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/ 2022-11-27T16:59:34.989 INFO:teuthology.task.install.rpm:Package version is 15.2.17-0 2022-11-27T16:59:34.994 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:59:34.994 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T16:59:34.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:59:35.009 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/ 2022-11-27T16:59:35.010 INFO:teuthology.task.install.rpm:Package version is 15.2.17-0 2022-11-27T16:59:35.033 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2022-11-27T16:59:35.034 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T16:59:35.034 INFO:teuthology.packaging:ref: None 2022-11-27T16:59:35.035 INFO:teuthology.packaging:tag: None 2022-11-27T16:59:35.035 INFO:teuthology.packaging:branch: octopus 2022-11-27T16:59:35.035 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:59:35.035 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/octopus/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:59:35.041 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:59:35.041 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2022-11-27T16:59:35.041 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:59:35.052 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:59:35.053 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2022-11-27T16:59:35.053 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:59:35.069 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2022-11-27T16:59:35.069 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T16:59:35.070 INFO:teuthology.packaging:ref: None 2022-11-27T16:59:35.070 INFO:teuthology.packaging:tag: None 2022-11-27T16:59:35.070 INFO:teuthology.packaging:branch: octopus 2022-11-27T16:59:35.070 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:59:35.071 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/octopus/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:59:35.081 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2022-11-27T16:59:35.081 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T16:59:35.082 INFO:teuthology.packaging:ref: None 2022-11-27T16:59:35.082 INFO:teuthology.packaging:tag: None 2022-11-27T16:59:35.082 INFO:teuthology.packaging:branch: octopus 2022-11-27T16:59:35.083 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:59:35.083 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/octopus/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:59:35.101 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:59:35.138 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:59:35.150 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:59:35.180 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:59:35.217 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:59:35.228 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:59:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2022-11-27T16:59:35.248 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2022-11-27T16:59:35.284 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2022-11-27T16:59:35.285 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2022-11-27T16:59:35.295 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2022-11-27T16:59:35.297 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2022-11-27T16:59:35.507 INFO:teuthology.orchestra.run.smithi154.stdout:64 files removed 2022-11-27T16:59:35.536 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw 2022-11-27T16:59:35.543 INFO:teuthology.orchestra.run.smithi159.stdout:64 files removed 2022-11-27T16:59:35.555 INFO:teuthology.orchestra.run.smithi165.stdout:64 files removed 2022-11-27T16:59:35.572 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw 2022-11-27T16:59:35.584 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw 2022-11-27T16:59:35.974 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 906 kB/s | 68 kB 00:00 2022-11-27T16:59:36.013 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 1.0 MB/s | 68 kB 00:00 2022-11-27T16:59:36.052 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 594 kB/s | 68 kB 00:00 2022-11-27T16:59:36.066 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 253 kB/s | 15 kB 00:00 2022-11-27T16:59:36.113 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 223 kB/s | 15 kB 00:00 2022-11-27T16:59:36.150 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 29 kB/s | 1.7 kB 00:00 2022-11-27T16:59:36.169 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 178 kB/s | 15 kB 00:00 2022-11-27T16:59:36.237 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 17 kB/s | 1.7 kB 00:00 2022-11-27T16:59:36.270 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 23 kB/s | 1.7 kB 00:00 2022-11-27T16:59:37.131 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-8 - AppStream 31 MB/s | 26 MB 00:00 2022-11-27T16:59:37.233 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS-8 - AppStream 27 MB/s | 26 MB 00:00 2022-11-27T16:59:37.261 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-8 - AppStream 24 MB/s | 26 MB 00:01 2022-11-27T16:59:42.506 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-8 - BaseOS 31 MB/s | 26 MB 00:00 2022-11-27T16:59:42.594 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-8 - BaseOS 39 MB/s | 26 MB 00:00 2022-11-27T16:59:42.717 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS-8 - BaseOS 32 MB/s | 26 MB 00:00 2022-11-27T16:59:46.472 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-8 - Extras 182 kB/s | 18 kB 00:00 2022-11-27T16:59:46.601 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-8 - Extras 157 kB/s | 18 kB 00:00 2022-11-27T16:59:46.849 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-8 - PowerTools 20 MB/s | 5.3 MB 00:00 2022-11-27T16:59:46.857 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS-8 - Extras 142 kB/s | 18 kB 00:00 2022-11-27T16:59:46.970 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-8 - PowerTools 21 MB/s | 5.3 MB 00:00 2022-11-27T16:59:47.242 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS-8 - PowerTools 20 MB/s | 5.3 MB 00:00 2022-11-27T16:59:50.391 INFO:teuthology.orchestra.run.smithi159.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2022-11-27T16:59:50.628 INFO:teuthology.orchestra.run.smithi154.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2022-11-27T16:59:50.792 INFO:teuthology.orchestra.run.smithi165.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2022-11-27T16:59:51.948 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 13 MB 00:01 2022-11-27T16:59:52.253 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 13 MB 00:01 2022-11-27T16:59:52.377 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 13 MB 00:01 2022-11-27T16:59:55.156 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 418 kB/s | 24 kB 00:00 2022-11-27T16:59:55.462 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 322 kB/s | 24 kB 00:00 2022-11-27T16:59:55.593 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 445 kB/s | 24 kB 00:00 2022-11-27T16:59:58.396 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T16:59:58.399 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T16:59:58.399 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2022-11-27T16:59:58.400 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T16:59:58.400 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2022-11-27T16:59:58.400 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:15.2.17-0.el8 ceph 9.9 M 2022-11-27T16:59:58.400 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2022-11-27T16:59:58.401 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:15.2.17-0.el8 ceph 3.5 M 2022-11-27T16:59:58.401 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:15.2.17-0.el8 ceph 1.9 M 2022-11-27T16:59:58.401 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2022-11-27T16:59:58.402 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:15.2.17-0.el8 ceph 5.6 M 2022-11-27T16:59:58.402 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:15.2.17-0.el8 ceph 21 M 2022-11-27T16:59:58.402 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:15.2.17-0.el8 ceph 25 k 2022-11-27T16:59:58.403 INFO:teuthology.orchestra.run.smithi159.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2022-11-27T16:59:58.403 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:59:58.403 INFO:teuthology.orchestra.run.smithi159.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2022-11-27T16:59:58.403 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:15.2.17-0.el8 ceph 508 k 2022-11-27T16:59:58.404 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:59:58.404 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2022-11-27T16:59:58.404 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:15.2.17-0.el8 ceph 380 k 2022-11-27T16:59:58.404 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2022-11-27T16:59:58.405 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:15.2.17-0.el8 ceph 3.2 M 2022-11-27T16:59:58.405 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:59:58.405 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2022-11-27T16:59:58.405 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:15.2.17-0.el8 ceph 40 k 2022-11-27T16:59:58.406 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:15.2.17-0.el8 ceph 77 k 2022-11-27T16:59:58.406 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:15.2.17-0.el8 ceph 164 k 2022-11-27T16:59:58.406 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2022-11-27T16:59:58.406 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:15.2.17-0.el8 ceph 364 k 2022-11-27T16:59:58.407 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:15.2.17-0.el8 ceph 349 k 2022-11-27T16:59:58.407 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:15.2.17-0.el8 ceph 113 k 2022-11-27T16:59:58.407 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T16:59:58.407 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T16:59:58.408 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T16:59:58.408 INFO:teuthology.orchestra.run.smithi159.stdout:Install 22 Packages 2022-11-27T16:59:58.408 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 2 Packages 2022-11-27T16:59:58.409 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T16:59:58.410 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 48 M 2022-11-27T16:59:58.410 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T16:59:58.557 INFO:teuthology.orchestra.run.smithi159.stdout:(1/24): ceph-base-15.2.17-0.el8.x86_64.rpm 37 MB/s | 5.6 MB 00:00 2022-11-27T16:59:58.591 INFO:teuthology.orchestra.run.smithi159.stdout:(2/24): ceph-selinux-15.2.17-0.el8.x86_64.rpm 750 kB/s | 25 kB 00:00 2022-11-27T16:59:58.625 INFO:teuthology.orchestra.run.smithi159.stdout:(3/24): libcephfs2-15.2.17-0.el8.x86_64.rpm 15 MB/s | 508 kB 00:00 2022-11-27T16:59:58.659 INFO:teuthology.orchestra.run.smithi159.stdout:(4/24): libradosstriper1-15.2.17-0.el8.x86_64.r 11 MB/s | 380 kB 00:00 2022-11-27T16:59:58.722 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T16:59:58.724 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T16:59:58.724 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2022-11-27T16:59:58.725 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T16:59:58.725 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2022-11-27T16:59:58.726 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:15.2.17-0.el8 ceph 9.9 M 2022-11-27T16:59:58.726 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2022-11-27T16:59:58.726 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:15.2.17-0.el8 ceph 3.5 M 2022-11-27T16:59:58.727 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:15.2.17-0.el8 ceph 1.9 M 2022-11-27T16:59:58.727 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2022-11-27T16:59:58.728 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:15.2.17-0.el8 ceph 5.6 M 2022-11-27T16:59:58.728 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:15.2.17-0.el8 ceph 21 M 2022-11-27T16:59:58.728 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:15.2.17-0.el8 ceph 25 k 2022-11-27T16:59:58.729 INFO:teuthology.orchestra.run.smithi154.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2022-11-27T16:59:58.729 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:59:58.729 INFO:teuthology.orchestra.run.smithi154.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2022-11-27T16:59:58.729 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:15.2.17-0.el8 ceph 508 k 2022-11-27T16:59:58.730 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:59:58.730 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2022-11-27T16:59:58.730 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:15.2.17-0.el8 ceph 380 k 2022-11-27T16:59:58.730 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2022-11-27T16:59:58.731 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:15.2.17-0.el8 ceph 3.2 M 2022-11-27T16:59:58.731 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:59:58.731 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2022-11-27T16:59:58.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:15.2.17-0.el8 ceph 40 k 2022-11-27T16:59:58.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:15.2.17-0.el8 ceph 77 k 2022-11-27T16:59:58.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:15.2.17-0.el8 ceph 164 k 2022-11-27T16:59:58.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2022-11-27T16:59:58.732 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:15.2.17-0.el8 ceph 364 k 2022-11-27T16:59:58.733 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:15.2.17-0.el8 ceph 349 k 2022-11-27T16:59:58.733 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:15.2.17-0.el8 ceph 113 k 2022-11-27T16:59:58.733 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T16:59:58.733 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T16:59:58.733 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T16:59:58.734 INFO:teuthology.orchestra.run.smithi154.stdout:Install 22 Packages 2022-11-27T16:59:58.734 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 2 Packages 2022-11-27T16:59:58.734 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T16:59:58.735 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 48 M 2022-11-27T16:59:58.735 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T16:59:58.811 INFO:teuthology.orchestra.run.smithi159.stdout:(5/24): ceph-radosgw-15.2.17-0.el8.x86_64.rpm 25 MB/s | 9.9 MB 00:00 2022-11-27T16:59:58.827 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T16:59:58.830 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T16:59:58.830 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2022-11-27T16:59:58.830 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T16:59:58.831 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2022-11-27T16:59:58.831 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:15.2.17-0.el8 ceph 9.9 M 2022-11-27T16:59:58.831 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2022-11-27T16:59:58.831 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:15.2.17-0.el8 ceph 3.5 M 2022-11-27T16:59:58.832 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:15.2.17-0.el8 ceph 1.9 M 2022-11-27T16:59:58.832 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2022-11-27T16:59:58.832 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:15.2.17-0.el8 ceph 5.6 M 2022-11-27T16:59:58.832 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:15.2.17-0.el8 ceph 21 M 2022-11-27T16:59:58.833 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:15.2.17-0.el8 ceph 25 k 2022-11-27T16:59:58.833 INFO:teuthology.orchestra.run.smithi165.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2022-11-27T16:59:58.833 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:59:58.833 INFO:teuthology.orchestra.run.smithi165.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2022-11-27T16:59:58.833 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:15.2.17-0.el8 ceph 508 k 2022-11-27T16:59:58.834 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:59:58.834 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2022-11-27T16:59:58.834 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:15.2.17-0.el8 ceph 380 k 2022-11-27T16:59:58.834 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2022-11-27T16:59:58.835 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:15.2.17-0.el8 ceph 3.2 M 2022-11-27T16:59:58.835 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:59:58.835 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2022-11-27T16:59:58.835 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:15.2.17-0.el8 ceph 40 k 2022-11-27T16:59:58.836 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:15.2.17-0.el8 ceph 77 k 2022-11-27T16:59:58.836 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:15.2.17-0.el8 ceph 164 k 2022-11-27T16:59:58.836 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2022-11-27T16:59:58.836 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:15.2.17-0.el8 ceph 364 k 2022-11-27T16:59:58.836 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:15.2.17-0.el8 ceph 349 k 2022-11-27T16:59:58.837 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:15.2.17-0.el8 ceph 113 k 2022-11-27T16:59:58.837 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T16:59:58.837 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T16:59:58.837 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T16:59:58.838 INFO:teuthology.orchestra.run.smithi165.stdout:Install 22 Packages 2022-11-27T16:59:58.838 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 2 Packages 2022-11-27T16:59:58.838 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T16:59:58.839 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 48 M 2022-11-27T16:59:58.839 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T16:59:58.860 INFO:teuthology.orchestra.run.smithi159.stdout:(6/24): python3-ceph-argparse-15.2.17-0.el8.x86 788 kB/s | 40 kB 00:00 2022-11-27T16:59:58.871 INFO:teuthology.orchestra.run.smithi154.stdout:(1/24): ceph-base-15.2.17-0.el8.x86_64.rpm 39 MB/s | 5.6 MB 00:00 2022-11-27T16:59:58.905 INFO:teuthology.orchestra.run.smithi154.stdout:(2/24): ceph-selinux-15.2.17-0.el8.x86_64.rpm 773 kB/s | 25 kB 00:00 2022-11-27T16:59:58.926 INFO:teuthology.orchestra.run.smithi159.stdout:(7/24): librgw2-15.2.17-0.el8.x86_64.rpm 12 MB/s | 3.2 MB 00:00 2022-11-27T16:59:58.939 INFO:teuthology.orchestra.run.smithi154.stdout:(3/24): libcephfs2-15.2.17-0.el8.x86_64.rpm 15 MB/s | 508 kB 00:00 2022-11-27T16:59:58.951 INFO:teuthology.orchestra.run.smithi159.stdout:(8/24): python3-ceph-common-15.2.17-0.el8.x86_6 840 kB/s | 77 kB 00:00 2022-11-27T16:59:58.964 INFO:teuthology.orchestra.run.smithi154.stdout:(4/24): libradosstriper1-15.2.17-0.el8.x86_64.r 15 MB/s | 380 kB 00:00 2022-11-27T16:59:58.977 INFO:teuthology.orchestra.run.smithi159.stdout:(9/24): python3-cephfs-15.2.17-0.el8.x86_64.rpm 3.2 MB/s | 164 kB 00:00 2022-11-27T16:59:58.994 INFO:teuthology.orchestra.run.smithi165.stdout:(1/24): ceph-base-15.2.17-0.el8.x86_64.rpm 35 MB/s | 5.6 MB 00:00 2022-11-27T16:59:59.002 INFO:teuthology.orchestra.run.smithi159.stdout:(10/24): python3-rados-15.2.17-0.el8.x86_64.rpm 7.1 MB/s | 364 kB 00:00 2022-11-27T16:59:59.019 INFO:teuthology.orchestra.run.smithi165.stdout:(2/24): ceph-selinux-15.2.17-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2022-11-27T16:59:59.036 INFO:teuthology.orchestra.run.smithi159.stdout:(11/24): python3-rgw-15.2.17-0.el8.x86_64.rpm 3.3 MB/s | 113 kB 00:00 2022-11-27T16:59:59.053 INFO:teuthology.orchestra.run.smithi165.stdout:(3/24): libcephfs2-15.2.17-0.el8.x86_64.rpm 15 MB/s | 508 kB 00:00 2022-11-27T16:59:59.061 INFO:teuthology.orchestra.run.smithi159.stdout:(12/24): python3-rbd-15.2.17-0.el8.x86_64.rpm 4.1 MB/s | 349 kB 00:00 2022-11-27T16:59:59.087 INFO:teuthology.orchestra.run.smithi165.stdout:(4/24): libradosstriper1-15.2.17-0.el8.x86_64.r 11 MB/s | 380 kB 00:00 2022-11-27T16:59:59.114 INFO:teuthology.orchestra.run.smithi154.stdout:(5/24): ceph-radosgw-15.2.17-0.el8.x86_64.rpm 26 MB/s | 9.9 MB 00:00 2022-11-27T16:59:59.128 INFO:teuthology.orchestra.run.smithi159.stdout:(13/24): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2022-11-27T16:59:59.170 INFO:teuthology.orchestra.run.smithi165.stdout:(5/24): librgw2-15.2.17-0.el8.x86_64.rpm 39 MB/s | 3.2 MB 00:00 2022-11-27T16:59:59.172 INFO:teuthology.orchestra.run.smithi154.stdout:(6/24): librgw2-15.2.17-0.el8.x86_64.rpm 15 MB/s | 3.2 MB 00:00 2022-11-27T16:59:59.178 INFO:teuthology.orchestra.run.smithi159.stdout:(14/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2022-11-27T16:59:59.195 INFO:teuthology.orchestra.run.smithi159.stdout:(15/24): python3-prettytable-0.7.2-14.el8.noarc 657 kB/s | 44 kB 00:00 2022-11-27T16:59:59.196 INFO:teuthology.orchestra.run.smithi165.stdout:(6/24): python3-ceph-argparse-15.2.17-0.el8.x86 1.6 MB/s | 40 kB 00:00 2022-11-27T16:59:59.198 INFO:teuthology.orchestra.run.smithi154.stdout:(7/24): python3-ceph-argparse-15.2.17-0.el8.x86 478 kB/s | 40 kB 00:00 2022-11-27T16:59:59.212 INFO:teuthology.orchestra.run.smithi159.stdout:(16/24): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2022-11-27T16:59:59.215 INFO:teuthology.orchestra.run.smithi154.stdout:(8/24): python3-ceph-common-15.2.17-0.el8.x86_6 1.8 MB/s | 77 kB 00:00 2022-11-27T16:59:59.221 INFO:teuthology.orchestra.run.smithi165.stdout:(7/24): python3-ceph-common-15.2.17-0.el8.x86_6 3.0 MB/s | 77 kB 00:00 2022-11-27T16:59:59.240 INFO:teuthology.orchestra.run.smithi154.stdout:(9/24): python3-cephfs-15.2.17-0.el8.x86_64.rpm 3.8 MB/s | 164 kB 00:00 2022-11-27T16:59:59.280 INFO:teuthology.orchestra.run.smithi165.stdout:(8/24): python3-cephfs-15.2.17-0.el8.x86_64.rpm 2.8 MB/s | 164 kB 00:00 2022-11-27T16:59:59.299 INFO:teuthology.orchestra.run.smithi154.stdout:(10/24): python3-rados-15.2.17-0.el8.x86_64.rpm 4.2 MB/s | 364 kB 00:00 2022-11-27T16:59:59.313 INFO:teuthology.orchestra.run.smithi165.stdout:(9/24): python3-rados-15.2.17-0.el8.x86_64.rpm 11 MB/s | 364 kB 00:00 2022-11-27T16:59:59.324 INFO:teuthology.orchestra.run.smithi154.stdout:(11/24): python3-rbd-15.2.17-0.el8.x86_64.rpm 4.1 MB/s | 349 kB 00:00 2022-11-27T16:59:59.338 INFO:teuthology.orchestra.run.smithi165.stdout:(10/24): python3-rbd-15.2.17-0.el8.x86_64.rpm 14 MB/s | 349 kB 00:00 2022-11-27T16:59:59.341 INFO:teuthology.orchestra.run.smithi154.stdout:(12/24): python3-rgw-15.2.17-0.el8.x86_64.rpm 2.6 MB/s | 113 kB 00:00 2022-11-27T16:59:59.363 INFO:teuthology.orchestra.run.smithi165.stdout:(11/24): python3-rgw-15.2.17-0.el8.x86_64.rpm 4.4 MB/s | 113 kB 00:00 2022-11-27T16:59:59.400 INFO:teuthology.orchestra.run.smithi154.stdout:(13/24): librdkafka-0.11.4-3.el8.x86_64.rpm 4.6 MB/s | 354 kB 00:00 2022-11-27T16:59:59.416 INFO:teuthology.orchestra.run.smithi154.stdout:(14/24): python3-prettytable-0.7.2-14.el8.noarc 2.6 MB/s | 44 kB 00:00 2022-11-27T16:59:59.421 INFO:teuthology.orchestra.run.smithi159.stdout:(17/24): ceph-common-15.2.17-0.el8.x86_64.rpm 20 MB/s | 21 MB 00:01 2022-11-27T16:59:59.434 INFO:teuthology.orchestra.run.smithi154.stdout:(15/24): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.7 MB/s | 47 kB 00:00 2022-11-27T16:59:59.459 INFO:teuthology.orchestra.run.smithi154.stdout:(16/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2022-11-27T16:59:59.471 INFO:teuthology.orchestra.run.smithi159.stdout:(18/24): fmt-6.2.1-1.el8.x86_64.rpm 422 kB/s | 116 kB 00:00 2022-11-27T16:59:59.483 INFO:teuthology.orchestra.run.smithi165.stdout:(12/24): ceph-radosgw-15.2.17-0.el8.x86_64.rpm 15 MB/s | 9.9 MB 00:00 2022-11-27T16:59:59.521 INFO:teuthology.orchestra.run.smithi159.stdout:(19/24): liboath-2.6.2-3.el8.x86_64.rpm 1.2 MB/s | 59 kB 00:00 2022-11-27T16:59:59.546 INFO:teuthology.orchestra.run.smithi165.stdout:(13/24): librdkafka-0.11.4-3.el8.x86_64.rpm 1.9 MB/s | 354 kB 00:00 2022-11-27T16:59:59.572 INFO:teuthology.orchestra.run.smithi159.stdout:(20/24): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2022-11-27T16:59:59.581 INFO:teuthology.orchestra.run.smithi165.stdout:(14/24): python3-prettytable-0.7.2-14.el8.noarc 1.3 MB/s | 44 kB 00:00 2022-11-27T16:59:59.597 INFO:teuthology.orchestra.run.smithi159.stdout:(21/24): gperftools-libs-2.7-9.el8.x86_64.rpm 796 kB/s | 306 kB 00:00 2022-11-27T16:59:59.606 INFO:teuthology.orchestra.run.smithi165.stdout:(15/24): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2022-11-27T16:59:59.631 INFO:teuthology.orchestra.run.smithi165.stdout:(16/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2022-11-27T16:59:59.647 INFO:teuthology.orchestra.run.smithi159.stdout:(22/24): librbd1-15.2.17-0.el8.x86_64.rpm 39 MB/s | 1.9 MB 00:00 2022-11-27T16:59:59.660 INFO:teuthology.orchestra.run.smithi154.stdout:(17/24): ceph-common-15.2.17-0.el8.x86_64.rpm 22 MB/s | 21 MB 00:00 2022-11-27T16:59:59.706 INFO:teuthology.orchestra.run.smithi159.stdout:(23/24): librados2-15.2.17-0.el8.x86_64.rpm 26 MB/s | 3.5 MB 00:00 2022-11-27T16:59:59.752 INFO:teuthology.orchestra.run.smithi159.stdout:(24/24): leveldb-1.22-1.el8.x86_64.rpm 547 kB/s | 181 kB 00:00 2022-11-27T16:59:59.756 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:59:59.756 INFO:teuthology.orchestra.run.smithi159.stdout:Total 36 MB/s | 48 MB 00:01 2022-11-27T16:59:59.785 INFO:teuthology.orchestra.run.smithi154.stdout:(18/24): fmt-6.2.1-1.el8.x86_64.rpm 332 kB/s | 116 kB 00:00 2022-11-27T16:59:59.795 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T16:59:59.810 INFO:teuthology.orchestra.run.smithi154.stdout:(19/24): leveldb-1.22-1.el8.x86_64.rpm 1.2 MB/s | 181 kB 00:00 2022-11-27T16:59:59.835 INFO:teuthology.orchestra.run.smithi154.stdout:(20/24): gperftools-libs-2.7-9.el8.x86_64.rpm 814 kB/s | 306 kB 00:00 2022-11-27T16:59:59.840 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T16:59:59.840 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T16:59:59.852 INFO:teuthology.orchestra.run.smithi154.stdout:(21/24): liboath-2.6.2-3.el8.x86_64.rpm 874 kB/s | 59 kB 00:00 2022-11-27T16:59:59.878 INFO:teuthology.orchestra.run.smithi154.stdout:(22/24): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2022-11-27T16:59:59.890 INFO:teuthology.orchestra.run.smithi165.stdout:(17/24): ceph-common-15.2.17-0.el8.x86_64.rpm 20 MB/s | 21 MB 00:01 2022-11-27T16:59:59.916 INFO:teuthology.orchestra.run.smithi165.stdout:(18/24): fmt-6.2.1-1.el8.x86_64.rpm 376 kB/s | 116 kB 00:00 2022-11-27T16:59:59.928 INFO:teuthology.orchestra.run.smithi154.stdout:(23/24): librbd1-15.2.17-0.el8.x86_64.rpm 26 MB/s | 1.9 MB 00:00 2022-11-27T16:59:59.941 INFO:teuthology.orchestra.run.smithi165.stdout:(19/24): gperftools-libs-2.7-9.el8.x86_64.rpm 991 kB/s | 306 kB 00:00 2022-11-27T16:59:59.966 INFO:teuthology.orchestra.run.smithi165.stdout:(20/24): liboath-2.6.2-3.el8.x86_64.rpm 1.1 MB/s | 59 kB 00:00 2022-11-27T16:59:59.991 INFO:teuthology.orchestra.run.smithi165.stdout:(21/24): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2022-11-27T17:00:00.006 INFO:teuthology.orchestra.run.smithi154.stdout:(24/24): librados2-15.2.17-0.el8.x86_64.rpm 21 MB/s | 3.5 MB 00:00 2022-11-27T17:00:00.010 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:00:00.010 INFO:teuthology.orchestra.run.smithi154.stdout:Total 38 MB/s | 48 MB 00:01 2022-11-27T17:00:00.046 INFO:teuthology.orchestra.run.smithi165.stdout:(22/24): librbd1-15.2.17-0.el8.x86_64.rpm 38 MB/s | 1.9 MB 00:00 2022-11-27T17:00:00.048 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T17:00:00.068 INFO:teuthology.orchestra.run.smithi165.stdout:(23/24): leveldb-1.22-1.el8.x86_64.rpm 1.0 MB/s | 181 kB 00:00 2022-11-27T17:00:00.093 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T17:00:00.094 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T17:00:00.125 INFO:teuthology.orchestra.run.smithi165.stdout:(24/24): librados2-15.2.17-0.el8.x86_64.rpm 22 MB/s | 3.5 MB 00:00 2022-11-27T17:00:00.129 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:00:00.130 INFO:teuthology.orchestra.run.smithi165.stdout:Total 37 MB/s | 48 MB 00:01 2022-11-27T17:00:00.157 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T17:00:00.160 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T17:00:00.169 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T17:00:00.214 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T17:00:00.215 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T17:00:00.391 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T17:00:00.394 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T17:00:00.512 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T17:00:00.515 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T17:00:01.168 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T17:00:01.363 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2022-11-27T17:00:01.433 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T17:00:01.534 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T17:00:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2022-11-27T17:00:01.643 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2022-11-27T17:00:01.709 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2022-11-27T17:00:01.816 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T17:00:01.873 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2022-11-27T17:00:01.929 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2022-11-27T17:00:02.051 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T17:00:02.138 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T17:00:02.191 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T17:00:02.383 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T17:00:02.415 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T17:00:02.495 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T17:00:02.609 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T17:00:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T17:00:02.727 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T17:00:02.748 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T17:00:02.845 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T17:00:02.966 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T17:00:02.974 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T17:00:02.999 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T17:00:03.210 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T17:00:03.226 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T17:00:03.235 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T17:00:03.434 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T17:00:03.449 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T17:00:03.461 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T17:00:03.511 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T17:00:03.677 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T17:00:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T17:00:03.739 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/26 2022-11-27T17:00:03.742 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T17:00:03.953 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T17:00:03.957 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/26 2022-11-27T17:00:04.022 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T17:00:04.075 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T17:00:04.124 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T17:00:04.209 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/26 2022-11-27T17:00:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T17:00:04.424 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T17:00:04.459 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T17:00:04.634 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T17:00:04.664 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T17:00:04.674 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T17:00:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T17:00:04.735 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T17:00:04.836 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T17:00:04.895 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T17:00:04.937 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2022-11-27T17:00:04.955 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2022-11-27T17:00:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T17:00:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T17:00:05.099 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2022-11-27T17:00:05.220 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T17:00:05.296 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T17:00:05.319 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T17:00:05.406 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T17:00:05.548 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:15.2.17-0.el8.x86_64 14/26 2022-11-27T17:00:05.549 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:15.2.17-0.el8.x86_64 14/26 2022-11-27T17:00:05.602 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:15.2.17-0.el8.x86_64 14/26 2022-11-27T17:00:05.747 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:15.2.17-0.el8.x86_64 15/26 2022-11-27T17:00:05.776 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:15.2.17-0.el8.x86_64 15/26 2022-11-27T17:00:05.802 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:15.2.17-0.el8.x86_64 15/26 2022-11-27T17:00:05.915 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T17:00:05.958 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T17:00:05.977 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T17:00:05.980 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T17:00:06.012 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T17:00:06.050 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T17:00:06.198 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2022-11-27T17:00:06.225 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2022-11-27T17:00:06.270 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2022-11-27T17:00:06.382 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/26 2022-11-27T17:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/26 2022-11-27T17:00:06.469 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/26 2022-11-27T17:00:06.597 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2022-11-27T17:00:06.641 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2022-11-27T17:00:06.653 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2022-11-27T17:00:06.791 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:15.2.17-0.el8.x86_64 20/26 2022-11-27T17:00:06.817 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:15.2.17-0.el8.x86_64 20/26 2022-11-27T17:00:06.828 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:15.2.17-0.el8.x86_64 20/26 2022-11-27T17:00:09.939 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T17:00:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T17:00:10.090 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T17:00:10.140 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T17:00:10.283 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T17:00:10.284 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T17:00:10.769 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T17:00:10.927 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T17:00:10.940 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T17:00:11.071 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T17:00:11.160 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T17:00:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T17:00:11.239 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T17:00:11.317 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T17:00:11.323 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T17:00:11.325 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T17:00:11.501 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T17:00:11.756 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T17:00:30.032 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T17:00:30.033 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2022-11-27T17:00:30.033 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2022-11-27T17:00:30.033 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /dev 2022-11-27T17:00:30.034 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /run 2022-11-27T17:00:30.034 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2022-11-27T17:00:30.034 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2022-11-27T17:00:30.034 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2022-11-27T17:00:30.034 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2022-11-27T17:00:30.035 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /dev 2022-11-27T17:00:30.035 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:30.243 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T17:00:30.244 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2022-11-27T17:00:30.244 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2022-11-27T17:00:30.244 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /dev 2022-11-27T17:00:30.245 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /run 2022-11-27T17:00:30.245 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2022-11-27T17:00:30.245 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2022-11-27T17:00:30.246 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2022-11-27T17:00:30.246 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2022-11-27T17:00:30.246 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /dev 2022-11-27T17:00:30.246 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:30.716 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T17:00:30.716 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2022-11-27T17:00:30.717 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2022-11-27T17:00:30.717 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /dev 2022-11-27T17:00:30.717 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /run 2022-11-27T17:00:30.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2022-11-27T17:00:30.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2022-11-27T17:00:30.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2022-11-27T17:00:30.718 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2022-11-27T17:00:30.719 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /dev 2022-11-27T17:00:30.719 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:31.092 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T17:00:31.163 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T17:00:31.163 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T17:00:31.299 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T17:00:31.374 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T17:00:31.375 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T17:00:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T17:00:31.414 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T17:00:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T17:00:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T17:00:31.778 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T17:00:31.851 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T17:00:31.851 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T17:00:32.068 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T17:00:32.068 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:15.2.17-0.el8.x86_64 1/26 2022-11-27T17:00:32.068 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:15.2.17-0.el8.x86_64 2/26 2022-11-27T17:00:32.068 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:15.2.17-0.el8.x86_64 3/26 2022-11-27T17:00:32.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T17:00:32.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T17:00:32.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T17:00:32.069 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T17:00:32.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 8/26 2022-11-27T17:00:32.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:15.2.17-0.el8.x86_64 9/26 2022-11-27T17:00:32.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T17:00:32.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:15.2.17-0.el8.x86_64 11/26 2022-11-27T17:00:32.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T17:00:32.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T17:00:32.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/26 2022-11-27T17:00:32.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2022-11-27T17:00:32.071 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2022-11-27T17:00:32.072 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/26 2022-11-27T17:00:32.072 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2022-11-27T17:00:32.072 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 19/26 2022-11-27T17:00:32.073 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2022-11-27T17:00:32.073 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2022-11-27T17:00:32.073 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2022-11-27T17:00:32.073 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T17:00:32.074 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2022-11-27T17:00:32.074 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:15.2.17-0.el8.x86_64 25/26 2022-11-27T17:00:32.090 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T17:00:32.090 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T17:00:32.274 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T17:00:32.274 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:15.2.17-0.el8.x86_64 1/26 2022-11-27T17:00:32.274 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:15.2.17-0.el8.x86_64 2/26 2022-11-27T17:00:32.275 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:15.2.17-0.el8.x86_64 3/26 2022-11-27T17:00:32.275 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T17:00:32.275 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T17:00:32.275 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T17:00:32.276 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T17:00:32.276 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 8/26 2022-11-27T17:00:32.276 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:15.2.17-0.el8.x86_64 9/26 2022-11-27T17:00:32.276 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T17:00:32.277 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:15.2.17-0.el8.x86_64 11/26 2022-11-27T17:00:32.277 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T17:00:32.277 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T17:00:32.277 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/26 2022-11-27T17:00:32.278 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2022-11-27T17:00:32.278 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2022-11-27T17:00:32.278 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/26 2022-11-27T17:00:32.278 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2022-11-27T17:00:32.279 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 19/26 2022-11-27T17:00:32.279 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2022-11-27T17:00:32.279 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2022-11-27T17:00:32.279 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2022-11-27T17:00:32.280 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T17:00:32.280 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2022-11-27T17:00:32.281 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:15.2.17-0.el8.x86_64 25/26 2022-11-27T17:00:32.772 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T17:00:32.773 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:15.2.17-0.el8.x86_64 1/26 2022-11-27T17:00:32.773 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:15.2.17-0.el8.x86_64 2/26 2022-11-27T17:00:32.773 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:15.2.17-0.el8.x86_64 3/26 2022-11-27T17:00:32.774 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T17:00:32.774 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T17:00:32.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T17:00:32.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T17:00:32.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 8/26 2022-11-27T17:00:32.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:15.2.17-0.el8.x86_64 9/26 2022-11-27T17:00:32.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T17:00:32.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:15.2.17-0.el8.x86_64 11/26 2022-11-27T17:00:32.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T17:00:32.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T17:00:32.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/26 2022-11-27T17:00:32.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2022-11-27T17:00:32.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2022-11-27T17:00:32.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/26 2022-11-27T17:00:32.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2022-11-27T17:00:32.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 19/26 2022-11-27T17:00:32.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2022-11-27T17:00:32.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2022-11-27T17:00:32.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2022-11-27T17:00:32.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T17:00:32.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2022-11-27T17:00:32.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:15.2.17-0.el8.x86_64 25/26 2022-11-27T17:00:33.022 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T17:00:33.022 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:33.023 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2022-11-27T17:00:33.023 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:15.2.17-0.el8.x86_64 librbd1-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.023 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2022-11-27T17:00:33.024 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.024 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.024 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.024 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.025 INFO:teuthology.orchestra.run.smithi159.stdout: fmt-6.2.1-1.el8.x86_64 2022-11-27T17:00:33.025 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T17:00:33.026 INFO:teuthology.orchestra.run.smithi159.stdout: leveldb-1.22-1.el8.x86_64 2022-11-27T17:00:33.026 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.027 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T17:00:33.027 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T17:00:33.027 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.028 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T17:00:33.028 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.029 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T17:00:33.029 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T17:00:33.030 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.030 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.031 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.031 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T17:00:33.032 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.032 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.032 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.033 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:33.033 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:00:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T17:00:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:33.164 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2022-11-27T17:00:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:15.2.17-0.el8.x86_64 librbd1-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.165 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2022-11-27T17:00:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: fmt-6.2.1-1.el8.x86_64 2022-11-27T17:00:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T17:00:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: leveldb-1.22-1.el8.x86_64 2022-11-27T17:00:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T17:00:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T17:00:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T17:00:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T17:00:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T17:00:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T17:00:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:33.171 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:00:33.312 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-test 2022-11-27T17:00:33.433 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-test 2022-11-27T17:00:33.713 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T17:00:33.714 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:33.714 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2022-11-27T17:00:33.714 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:15.2.17-0.el8.x86_64 librbd1-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.714 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2022-11-27T17:00:33.715 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.715 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.715 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.716 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.716 INFO:teuthology.orchestra.run.smithi154.stdout: fmt-6.2.1-1.el8.x86_64 2022-11-27T17:00:33.716 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T17:00:33.716 INFO:teuthology.orchestra.run.smithi154.stdout: leveldb-1.22-1.el8.x86_64 2022-11-27T17:00:33.717 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.717 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T17:00:33.717 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T17:00:33.718 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.718 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T17:00:33.718 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.718 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T17:00:33.718 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T17:00:33.719 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.719 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.719 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.719 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T17:00:33.720 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.720 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.720 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:33.720 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:33.721 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:00:33.922 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:38 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:00:34.020 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-test 2022-11-27T17:00:34.042 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:39 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:00:34.634 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:39 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:00:34.975 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:00:34.976 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:00:34.976 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2022-11-27T17:00:34.977 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:00:34.977 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2022-11-27T17:00:34.977 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:15.2.17-0.el8 ceph 42 M 2022-11-27T17:00:34.977 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2022-11-27T17:00:34.978 INFO:teuthology.orchestra.run.smithi159.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2022-11-27T17:00:34.978 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2022-11-27T17:00:34.978 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2022-11-27T17:00:34.978 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2022-11-27T17:00:34.978 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:34.979 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T17:00:34.979 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:00:34.979 INFO:teuthology.orchestra.run.smithi159.stdout:Install 5 Packages 2022-11-27T17:00:34.979 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:34.980 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 43 M 2022-11-27T17:00:34.980 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 155 M 2022-11-27T17:00:34.981 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T17:00:35.055 INFO:teuthology.orchestra.run.smithi159.stdout:(1/5): jq-1.6-6.el8.x86_64.rpm 2.6 MB/s | 202 kB 00:00 2022-11-27T17:00:35.081 INFO:teuthology.orchestra.run.smithi159.stdout:(2/5): oniguruma-6.8.2-2.el8.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2022-11-27T17:00:35.097 INFO:teuthology.orchestra.run.smithi159.stdout:(3/5): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.2 MB/s | 71 kB 00:00 2022-11-27T17:00:35.123 INFO:teuthology.orchestra.run.smithi159.stdout:(4/5): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2022-11-27T17:00:35.150 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:00:35.151 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:00:35.151 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2022-11-27T17:00:35.152 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:00:35.152 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2022-11-27T17:00:35.152 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:15.2.17-0.el8 ceph 42 M 2022-11-27T17:00:35.152 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2022-11-27T17:00:35.153 INFO:teuthology.orchestra.run.smithi165.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2022-11-27T17:00:35.153 INFO:teuthology.orchestra.run.smithi165.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2022-11-27T17:00:35.153 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2022-11-27T17:00:35.153 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2022-11-27T17:00:35.154 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:35.154 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T17:00:35.154 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:00:35.154 INFO:teuthology.orchestra.run.smithi165.stdout:Install 5 Packages 2022-11-27T17:00:35.155 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:35.157 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 43 M 2022-11-27T17:00:35.157 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 155 M 2022-11-27T17:00:35.158 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T17:00:35.232 INFO:teuthology.orchestra.run.smithi165.stdout:(1/5): jq-1.6-6.el8.x86_64.rpm 2.5 MB/s | 202 kB 00:00 2022-11-27T17:00:35.258 INFO:teuthology.orchestra.run.smithi165.stdout:(2/5): oniguruma-6.8.2-2.el8.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2022-11-27T17:00:35.283 INFO:teuthology.orchestra.run.smithi165.stdout:(3/5): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 71 kB 00:00 2022-11-27T17:00:35.337 INFO:teuthology.orchestra.run.smithi165.stdout:(4/5): socat-1.7.4.1-1.el8.x86_64.rpm 3.0 MB/s | 323 kB 00:00 2022-11-27T17:00:35.648 INFO:teuthology.orchestra.run.smithi159.stdout:(5/5): ceph-test-15.2.17-0.el8.x86_64.rpm 63 MB/s | 42 MB 00:00 2022-11-27T17:00:35.651 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:00:35.651 INFO:teuthology.orchestra.run.smithi159.stdout:Total 64 MB/s | 43 MB 00:00 2022-11-27T17:00:35.693 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T17:00:35.713 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T17:00:35.713 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T17:00:35.726 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:00:35.727 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:00:35.728 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2022-11-27T17:00:35.728 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:00:35.728 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2022-11-27T17:00:35.728 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:15.2.17-0.el8 ceph 42 M 2022-11-27T17:00:35.729 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2022-11-27T17:00:35.729 INFO:teuthology.orchestra.run.smithi154.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2022-11-27T17:00:35.729 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2022-11-27T17:00:35.729 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2022-11-27T17:00:35.730 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2022-11-27T17:00:35.730 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:35.730 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T17:00:35.730 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:00:35.730 INFO:teuthology.orchestra.run.smithi154.stdout:Install 5 Packages 2022-11-27T17:00:35.731 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:35.732 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 43 M 2022-11-27T17:00:35.732 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 155 M 2022-11-27T17:00:35.732 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T17:00:35.800 INFO:teuthology.orchestra.run.smithi165.stdout:(5/5): ceph-test-15.2.17-0.el8.x86_64.rpm 65 MB/s | 42 MB 00:00 2022-11-27T17:00:35.801 INFO:teuthology.orchestra.run.smithi154.stdout:(1/5): jq-1.6-6.el8.x86_64.rpm 2.8 MB/s | 202 kB 00:00 2022-11-27T17:00:35.803 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:00:35.803 INFO:teuthology.orchestra.run.smithi165.stdout:Total 66 MB/s | 43 MB 00:00 2022-11-27T17:00:35.845 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T17:00:35.860 INFO:teuthology.orchestra.run.smithi154.stdout:(2/5): oniguruma-6.8.2-2.el8.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2022-11-27T17:00:35.864 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T17:00:35.864 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T17:00:35.885 INFO:teuthology.orchestra.run.smithi154.stdout:(3/5): socat-1.7.4.1-1.el8.x86_64.rpm 3.8 MB/s | 323 kB 00:00 2022-11-27T17:00:35.911 INFO:teuthology.orchestra.run.smithi154.stdout:(4/5): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2022-11-27T17:00:35.967 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T17:00:35.970 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T17:00:36.123 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T17:00:36.126 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T17:00:36.358 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T17:00:36.411 INFO:teuthology.orchestra.run.smithi154.stdout:(5/5): ceph-test-15.2.17-0.el8.x86_64.rpm 62 MB/s | 42 MB 00:00 2022-11-27T17:00:36.413 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:00:36.414 INFO:teuthology.orchestra.run.smithi154.stdout:Total 63 MB/s | 43 MB 00:00 2022-11-27T17:00:36.454 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T17:00:36.472 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T17:00:36.472 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T17:00:36.548 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/5 2022-11-27T17:00:36.550 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T17:00:36.722 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/5 2022-11-27T17:00:36.727 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T17:00:36.730 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T17:00:36.758 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/5 2022-11-27T17:00:36.885 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T17:00:36.953 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T17:00:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/5 2022-11-27T17:00:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T17:00:37.146 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T17:00:37.149 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T17:00:37.331 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/5 2022-11-27T17:00:37.497 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/5 2022-11-27T17:00:37.661 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T17:00:37.742 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T17:00:40.822 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : jq-1.6-6.el8.x86_64 4/5 2022-11-27T17:00:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : jq-1.6-6.el8.x86_64 4/5 2022-11-27T17:00:41.066 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T17:00:41.231 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T17:00:41.316 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T17:00:41.316 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:15.2.17-0.el8.x86_64 1/5 2022-11-27T17:00:41.317 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : jq-1.6-6.el8.x86_64 2/5 2022-11-27T17:00:41.317 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T17:00:41.317 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/5 2022-11-27T17:00:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T17:00:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:15.2.17-0.el8.x86_64 1/5 2022-11-27T17:00:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : jq-1.6-6.el8.x86_64 2/5 2022-11-27T17:00:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T17:00:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/5 2022-11-27T17:00:41.587 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : jq-1.6-6.el8.x86_64 4/5 2022-11-27T17:00:41.766 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 5/5 2022-11-27T17:00:41.766 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:41.766 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2022-11-27T17:00:41.767 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:15.2.17-0.el8.x86_64 jq-1.6-6.el8.x86_64 2022-11-27T17:00:41.767 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma-6.8.2-2.el8.x86_64 socat-1.7.4.1-1.el8.x86_64 2022-11-27T17:00:41.767 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T17:00:41.767 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:41.768 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:00:41.825 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T17:00:41.960 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 5/5 2022-11-27T17:00:41.960 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:41.960 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2022-11-27T17:00:41.961 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:15.2.17-0.el8.x86_64 jq-1.6-6.el8.x86_64 2022-11-27T17:00:41.961 INFO:teuthology.orchestra.run.smithi165.stdout: oniguruma-6.8.2-2.el8.x86_64 socat-1.7.4.1-1.el8.x86_64 2022-11-27T17:00:41.961 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T17:00:41.961 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:41.962 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:00:42.027 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph 2022-11-27T17:00:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T17:00:42.070 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:15.2.17-0.el8.x86_64 1/5 2022-11-27T17:00:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : jq-1.6-6.el8.x86_64 2/5 2022-11-27T17:00:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T17:00:42.071 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/5 2022-11-27T17:00:42.236 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph 2022-11-27T17:00:42.534 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 5/5 2022-11-27T17:00:42.534 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:42.535 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2022-11-27T17:00:42.535 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:15.2.17-0.el8.x86_64 jq-1.6-6.el8.x86_64 2022-11-27T17:00:42.535 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma-6.8.2-2.el8.x86_64 socat-1.7.4.1-1.el8.x86_64 2022-11-27T17:00:42.535 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T17:00:42.535 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:42.536 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:00:42.633 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:00:47 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:00:42.780 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph 2022-11-27T17:00:42.847 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:47 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:00:43.388 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:48 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:00:43.711 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:00:43.714 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:00:43.715 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2022-11-27T17:00:43.715 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:00:43.715 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2022-11-27T17:00:43.715 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:15.2.17-0.el8 ceph 6.3 k 2022-11-27T17:00:43.716 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2022-11-27T17:00:43.716 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:15.2.17-0.el8 ceph 2.0 M 2022-11-27T17:00:43.716 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:15.2.17-0.el8 ceph 1.4 M 2022-11-27T17:00:43.716 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:15.2.17-0.el8 ceph-noarch 193 k 2022-11-27T17:00:43.717 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:15.2.17-0.el8 ceph 4.3 M 2022-11-27T17:00:43.717 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:15.2.17-0.el8 ceph 17 M 2022-11-27T17:00:43.717 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T17:00:43.717 INFO:teuthology.orchestra.run.smithi159.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T17:00:43.718 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2022-11-27T17:00:43.718 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T17:00:43.718 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T17:00:43.718 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2022-11-27T17:00:43.719 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T17:00:43.719 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T17:00:43.719 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T17:00:43.719 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2022-11-27T17:00:43.719 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2022-11-27T17:00:43.720 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T17:00:43.720 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T17:00:43.720 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T17:00:43.720 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2022-11-27T17:00:43.721 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2022-11-27T17:00:43.721 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2022-11-27T17:00:43.721 INFO:teuthology.orchestra.run.smithi159.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T17:00:43.721 INFO:teuthology.orchestra.run.smithi159.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T17:00:43.722 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T17:00:43.722 INFO:teuthology.orchestra.run.smithi159.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T17:00:43.722 INFO:teuthology.orchestra.run.smithi159.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T17:00:43.722 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T17:00:43.722 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T17:00:43.723 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2022-11-27T17:00:43.723 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T17:00:43.723 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:43.723 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T17:00:43.724 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:00:43.724 INFO:teuthology.orchestra.run.smithi159.stdout:Install 32 Packages 2022-11-27T17:00:43.724 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:43.725 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 28 M 2022-11-27T17:00:43.725 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 91 M 2022-11-27T17:00:43.725 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T17:00:43.781 INFO:teuthology.orchestra.run.smithi159.stdout:(1/32): ceph-15.2.17-0.el8.x86_64.rpm 106 kB/s | 6.3 kB 00:00 2022-11-27T17:00:43.823 INFO:teuthology.orchestra.run.smithi159.stdout:(2/32): ceph-mds-15.2.17-0.el8.x86_64.rpm 20 MB/s | 2.0 MB 00:00 2022-11-27T17:00:43.856 INFO:teuthology.orchestra.run.smithi159.stdout:(3/32): ceph-mgr-15.2.17-0.el8.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2022-11-27T17:00:43.881 INFO:teuthology.orchestra.run.smithi159.stdout:(4/32): ceph-mgr-modules-core-15.2.17-0.el8.noa 7.6 MB/s | 193 kB 00:00 2022-11-27T17:00:43.938 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:00:43.942 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:00:43.942 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2022-11-27T17:00:43.942 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:00:43.943 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2022-11-27T17:00:43.943 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:15.2.17-0.el8 ceph 6.3 k 2022-11-27T17:00:43.943 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2022-11-27T17:00:43.943 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:15.2.17-0.el8 ceph 2.0 M 2022-11-27T17:00:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:15.2.17-0.el8 ceph 1.4 M 2022-11-27T17:00:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:15.2.17-0.el8 ceph-noarch 193 k 2022-11-27T17:00:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:15.2.17-0.el8 ceph 4.3 M 2022-11-27T17:00:43.944 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:15.2.17-0.el8 ceph 17 M 2022-11-27T17:00:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T17:00:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T17:00:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2022-11-27T17:00:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T17:00:43.945 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T17:00:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2022-11-27T17:00:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T17:00:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T17:00:43.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T17:00:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2022-11-27T17:00:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2022-11-27T17:00:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T17:00:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T17:00:43.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T17:00:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2022-11-27T17:00:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2022-11-27T17:00:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2022-11-27T17:00:43.948 INFO:teuthology.orchestra.run.smithi165.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T17:00:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T17:00:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T17:00:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T17:00:43.949 INFO:teuthology.orchestra.run.smithi165.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T17:00:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T17:00:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T17:00:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2022-11-27T17:00:43.950 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T17:00:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:43.951 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T17:00:43.951 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:00:43.951 INFO:teuthology.orchestra.run.smithi165.stdout:Install 32 Packages 2022-11-27T17:00:43.951 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:43.952 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 28 M 2022-11-27T17:00:43.953 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 91 M 2022-11-27T17:00:43.953 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T17:00:43.982 INFO:teuthology.orchestra.run.smithi159.stdout:(5/32): ceph-mon-15.2.17-0.el8.x86_64.rpm 22 MB/s | 4.3 MB 00:00 2022-11-27T17:00:43.998 INFO:teuthology.orchestra.run.smithi165.stdout:(1/32): ceph-15.2.17-0.el8.x86_64.rpm 127 kB/s | 6.3 kB 00:00 2022-11-27T17:00:44.015 INFO:teuthology.orchestra.run.smithi159.stdout:(6/32): python3-mako-1.0.6-13.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2022-11-27T17:00:44.040 INFO:teuthology.orchestra.run.smithi165.stdout:(2/32): ceph-mds-15.2.17-0.el8.x86_64.rpm 22 MB/s | 2.0 MB 00:00 2022-11-27T17:00:44.041 INFO:teuthology.orchestra.run.smithi159.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.1 MB/s | 103 kB 00:00 2022-11-27T17:00:44.066 INFO:teuthology.orchestra.run.smithi159.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 466 kB/s | 39 kB 00:00 2022-11-27T17:00:44.081 INFO:teuthology.orchestra.run.smithi165.stdout:(3/32): ceph-mgr-15.2.17-0.el8.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2022-11-27T17:00:44.091 INFO:teuthology.orchestra.run.smithi159.stdout:(9/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2022-11-27T17:00:44.107 INFO:teuthology.orchestra.run.smithi165.stdout:(4/32): ceph-mgr-modules-core-15.2.17-0.el8.noa 7.5 MB/s | 193 kB 00:00 2022-11-27T17:00:44.134 INFO:teuthology.orchestra.run.smithi159.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.5 MB/s | 237 kB 00:00 2022-11-27T17:00:44.192 INFO:teuthology.orchestra.run.smithi159.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.6 MB/s | 457 kB 00:00 2022-11-27T17:00:44.209 INFO:teuthology.orchestra.run.smithi165.stdout:(5/32): ceph-mon-15.2.17-0.el8.x86_64.rpm 21 MB/s | 4.3 MB 00:00 2022-11-27T17:00:44.244 INFO:teuthology.orchestra.run.smithi159.stdout:(12/32): python3-cryptography-3.2.1-5.el8.x86_6 5.9 MB/s | 559 kB 00:00 2022-11-27T17:00:44.274 INFO:teuthology.orchestra.run.smithi165.stdout:(6/32): python3-mako-1.0.6-13.el8.noarch.rpm 940 kB/s | 157 kB 00:00 2022-11-27T17:00:44.316 INFO:teuthology.orchestra.run.smithi165.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 2.5 MB/s | 103 kB 00:00 2022-11-27T17:00:44.341 INFO:teuthology.orchestra.run.smithi165.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 295 kB/s | 39 kB 00:00 2022-11-27T17:00:44.366 INFO:teuthology.orchestra.run.smithi165.stdout:(9/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2022-11-27T17:00:44.410 INFO:teuthology.orchestra.run.smithi159.stdout:(13/32): ceph-osd-15.2.17-0.el8.x86_64.rpm 29 MB/s | 17 MB 00:00 2022-11-27T17:00:44.435 INFO:teuthology.orchestra.run.smithi159.stdout:(14/32): python3-pycparser-2.14-14.el8.noarch.r 450 kB/s | 109 kB 00:00 2022-11-27T17:00:44.465 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:00:44.468 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:00:44.469 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2022-11-27T17:00:44.469 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:00:44.469 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2022-11-27T17:00:44.469 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:15.2.17-0.el8 ceph 6.3 k 2022-11-27T17:00:44.470 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2022-11-27T17:00:44.470 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:15.2.17-0.el8 ceph 2.0 M 2022-11-27T17:00:44.470 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:15.2.17-0.el8 ceph 1.4 M 2022-11-27T17:00:44.470 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:15.2.17-0.el8 ceph-noarch 193 k 2022-11-27T17:00:44.470 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:15.2.17-0.el8 ceph 4.3 M 2022-11-27T17:00:44.471 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:15.2.17-0.el8 ceph 17 M 2022-11-27T17:00:44.471 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T17:00:44.471 INFO:teuthology.orchestra.run.smithi154.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T17:00:44.471 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2022-11-27T17:00:44.472 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T17:00:44.472 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T17:00:44.472 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2022-11-27T17:00:44.472 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T17:00:44.473 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T17:00:44.473 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T17:00:44.473 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2022-11-27T17:00:44.473 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2022-11-27T17:00:44.474 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T17:00:44.474 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T17:00:44.474 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T17:00:44.474 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2022-11-27T17:00:44.475 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2022-11-27T17:00:44.475 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2022-11-27T17:00:44.475 INFO:teuthology.orchestra.run.smithi154.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T17:00:44.475 INFO:teuthology.orchestra.run.smithi154.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T17:00:44.475 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T17:00:44.476 INFO:teuthology.orchestra.run.smithi154.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T17:00:44.476 INFO:teuthology.orchestra.run.smithi154.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T17:00:44.476 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T17:00:44.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T17:00:44.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2022-11-27T17:00:44.477 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T17:00:44.477 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:44.478 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T17:00:44.478 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:00:44.478 INFO:teuthology.orchestra.run.smithi154.stdout:Install 32 Packages 2022-11-27T17:00:44.479 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:44.480 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 28 M 2022-11-27T17:00:44.480 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 91 M 2022-11-27T17:00:44.481 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T17:00:44.518 INFO:teuthology.orchestra.run.smithi159.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 149 kB/s | 44 kB 00:00 2022-11-27T17:00:44.536 INFO:teuthology.orchestra.run.smithi154.stdout:(1/32): ceph-15.2.17-0.el8.x86_64.rpm 104 kB/s | 6.3 kB 00:00 2022-11-27T17:00:44.550 INFO:teuthology.orchestra.run.smithi165.stdout:(10/32): ceph-osd-15.2.17-0.el8.x86_64.rpm 33 MB/s | 17 MB 00:00 2022-11-27T17:00:44.575 INFO:teuthology.orchestra.run.smithi165.stdout:(11/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.1 MB/s | 237 kB 00:00 2022-11-27T17:00:44.585 INFO:teuthology.orchestra.run.smithi159.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.0 MB/s | 185 kB 00:00 2022-11-27T17:00:44.600 INFO:teuthology.orchestra.run.smithi165.stdout:(12/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.7 MB/s | 457 kB 00:00 2022-11-27T17:00:44.602 INFO:teuthology.orchestra.run.smithi159.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.0 MB/s | 173 kB 00:00 2022-11-27T17:00:44.620 INFO:teuthology.orchestra.run.smithi154.stdout:(2/32): ceph-mds-15.2.17-0.el8.x86_64.rpm 14 MB/s | 2.0 MB 00:00 2022-11-27T17:00:44.634 INFO:teuthology.orchestra.run.smithi165.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 1.8 MB/s | 109 kB 00:00 2022-11-27T17:00:44.636 INFO:teuthology.orchestra.run.smithi159.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2022-11-27T17:00:44.661 INFO:teuthology.orchestra.run.smithi159.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 301 kB/s | 18 kB 00:00 2022-11-27T17:00:44.661 INFO:teuthology.orchestra.run.smithi154.stdout:(3/32): ceph-mgr-15.2.17-0.el8.x86_64.rpm 7.7 MB/s | 1.4 MB 00:00 2022-11-27T17:00:44.667 INFO:teuthology.orchestra.run.smithi165.stdout:(14/32): python3-cryptography-3.2.1-5.el8.x86_6 4.7 MB/s | 559 kB 00:00 2022-11-27T17:00:44.686 INFO:teuthology.orchestra.run.smithi159.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.2 MB/s | 384 kB 00:00 2022-11-27T17:00:44.695 INFO:teuthology.orchestra.run.smithi154.stdout:(4/32): ceph-mgr-modules-core-15.2.17-0.el8.noa 5.7 MB/s | 193 kB 00:00 2022-11-27T17:00:44.712 INFO:teuthology.orchestra.run.smithi159.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 644 kB/s | 49 kB 00:00 2022-11-27T17:00:44.737 INFO:teuthology.orchestra.run.smithi159.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 776 kB/s | 59 kB 00:00 2022-11-27T17:00:44.762 INFO:teuthology.orchestra.run.smithi159.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2022-11-27T17:00:44.770 INFO:teuthology.orchestra.run.smithi154.stdout:(5/32): ceph-mon-15.2.17-0.el8.x86_64.rpm 19 MB/s | 4.3 MB 00:00 2022-11-27T17:00:44.784 INFO:teuthology.orchestra.run.smithi165.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 238 kB/s | 44 kB 00:00 2022-11-27T17:00:44.787 INFO:teuthology.orchestra.run.smithi159.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 206 kB/s | 16 kB 00:00 2022-11-27T17:00:44.813 INFO:teuthology.orchestra.run.smithi159.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 248 kB/s | 19 kB 00:00 2022-11-27T17:00:44.841 INFO:teuthology.orchestra.run.smithi165.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 897 kB/s | 185 kB 00:00 2022-11-27T17:00:44.858 INFO:teuthology.orchestra.run.smithi154.stdout:(6/32): python3-mako-1.0.6-13.el8.noarch.rpm 964 kB/s | 157 kB 00:00 2022-11-27T17:00:44.860 INFO:teuthology.orchestra.run.smithi159.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 246 kB/s | 24 kB 00:00 2022-11-27T17:00:44.868 INFO:teuthology.orchestra.run.smithi165.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 860 kB/s | 173 kB 00:00 2022-11-27T17:00:44.888 INFO:teuthology.orchestra.run.smithi159.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 291 kB/s | 29 kB 00:00 2022-11-27T17:00:44.893 INFO:teuthology.orchestra.run.smithi165.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 212 kB/s | 11 kB 00:00 2022-11-27T17:00:44.896 INFO:teuthology.orchestra.run.smithi154.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 2.7 MB/s | 103 kB 00:00 2022-11-27T17:00:44.914 INFO:teuthology.orchestra.run.smithi159.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 273 kB/s | 27 kB 00:00 2022-11-27T17:00:44.921 INFO:teuthology.orchestra.run.smithi154.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 259 kB/s | 39 kB 00:00 2022-11-27T17:00:44.927 INFO:teuthology.orchestra.run.smithi165.stdout:(19/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.6 MB/s | 384 kB 00:00 2022-11-27T17:00:44.939 INFO:teuthology.orchestra.run.smithi159.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 3.0 MB/s | 241 kB 00:00 2022-11-27T17:00:44.946 INFO:teuthology.orchestra.run.smithi154.stdout:(9/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2022-11-27T17:00:44.952 INFO:teuthology.orchestra.run.smithi165.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 209 kB/s | 18 kB 00:00 2022-11-27T17:00:44.964 INFO:teuthology.orchestra.run.smithi159.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2022-11-27T17:00:44.972 INFO:teuthology.orchestra.run.smithi154.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.3 MB/s | 237 kB 00:00 2022-11-27T17:00:44.977 INFO:teuthology.orchestra.run.smithi165.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 580 kB/s | 49 kB 00:00 2022-11-27T17:00:44.981 INFO:teuthology.orchestra.run.smithi159.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 548 kB/s | 23 kB 00:00 2022-11-27T17:00:45.003 INFO:teuthology.orchestra.run.smithi165.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 778 kB/s | 59 kB 00:00 2022-11-27T17:00:45.047 INFO:teuthology.orchestra.run.smithi165.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 265 kB/s | 16 kB 00:00 2022-11-27T17:00:45.048 INFO:teuthology.orchestra.run.smithi154.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 5.3 MB/s | 457 kB 00:00 2022-11-27T17:00:45.048 INFO:teuthology.orchestra.run.smithi154.stdout:(12/32): python3-cryptography-3.2.1-5.el8.x86_6 9.3 MB/s | 559 kB 00:00 2022-11-27T17:00:45.048 INFO:teuthology.orchestra.run.smithi159.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.7 MB/s | 251 kB 00:00 2022-11-27T17:00:45.049 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:00:45.049 INFO:teuthology.orchestra.run.smithi159.stdout:Total 22 MB/s | 28 MB 00:01 2022-11-27T17:00:45.054 INFO:teuthology.orchestra.run.smithi165.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 371 kB/s | 19 kB 00:00 2022-11-27T17:00:45.056 INFO:teuthology.orchestra.run.smithi154.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 2.1 MB/s | 109 kB 00:00 2022-11-27T17:00:45.079 INFO:teuthology.orchestra.run.smithi165.stdout:(25/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2022-11-27T17:00:45.093 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T17:00:45.095 INFO:teuthology.orchestra.run.smithi165.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 409 kB/s | 24 kB 00:00 2022-11-27T17:00:45.132 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T17:00:45.132 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T17:00:45.146 INFO:teuthology.orchestra.run.smithi165.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 317 kB/s | 29 kB 00:00 2022-11-27T17:00:45.163 INFO:teuthology.orchestra.run.smithi165.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 328 kB/s | 27 kB 00:00 2022-11-27T17:00:45.188 INFO:teuthology.orchestra.run.smithi165.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.5 MB/s | 241 kB 00:00 2022-11-27T17:00:45.213 INFO:teuthology.orchestra.run.smithi165.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 3.6 MB/s | 251 kB 00:00 2022-11-27T17:00:45.238 INFO:teuthology.orchestra.run.smithi165.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2022-11-27T17:00:45.240 INFO:teuthology.orchestra.run.smithi154.stdout:(14/32): ceph-osd-15.2.17-0.el8.x86_64.rpm 27 MB/s | 17 MB 00:00 2022-11-27T17:00:45.255 INFO:teuthology.orchestra.run.smithi165.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 342 kB/s | 23 kB 00:00 2022-11-27T17:00:45.259 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:00:45.259 INFO:teuthology.orchestra.run.smithi165.stdout:Total 22 MB/s | 28 MB 00:01 2022-11-27T17:00:45.334 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T17:00:45.357 INFO:teuthology.orchestra.run.smithi154.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 134 kB/s | 44 kB 00:00 2022-11-27T17:00:45.372 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T17:00:45.372 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T17:00:45.382 INFO:teuthology.orchestra.run.smithi154.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 569 kB/s | 185 kB 00:00 2022-11-27T17:00:45.407 INFO:teuthology.orchestra.run.smithi154.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.0 MB/s | 173 kB 00:00 2022-11-27T17:00:45.424 INFO:teuthology.orchestra.run.smithi154.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 264 kB/s | 11 kB 00:00 2022-11-27T17:00:45.431 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T17:00:45.435 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T17:00:45.449 INFO:teuthology.orchestra.run.smithi154.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 420 kB/s | 18 kB 00:00 2022-11-27T17:00:45.466 INFO:teuthology.orchestra.run.smithi154.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 1.1 MB/s | 49 kB 00:00 2022-11-27T17:00:45.491 INFO:teuthology.orchestra.run.smithi154.stdout:(21/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.8 MB/s | 384 kB 00:00 2022-11-27T17:00:45.508 INFO:teuthology.orchestra.run.smithi154.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 997 kB/s | 59 kB 00:00 2022-11-27T17:00:45.534 INFO:teuthology.orchestra.run.smithi154.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 371 kB/s | 16 kB 00:00 2022-11-27T17:00:45.587 INFO:teuthology.orchestra.run.smithi154.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.3 MB/s | 283 kB 00:00 2022-11-27T17:00:45.609 INFO:teuthology.orchestra.run.smithi154.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 186 kB/s | 19 kB 00:00 2022-11-27T17:00:45.626 INFO:teuthology.orchestra.run.smithi154.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 259 kB/s | 24 kB 00:00 2022-11-27T17:00:45.643 INFO:teuthology.orchestra.run.smithi154.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 521 kB/s | 29 kB 00:00 2022-11-27T17:00:45.665 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T17:00:45.668 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T17:00:45.669 INFO:teuthology.orchestra.run.smithi154.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 656 kB/s | 27 kB 00:00 2022-11-27T17:00:45.694 INFO:teuthology.orchestra.run.smithi154.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 4.9 MB/s | 251 kB 00:00 2022-11-27T17:00:45.719 INFO:teuthology.orchestra.run.smithi154.stdout:(30/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.5 MB/s | 241 kB 00:00 2022-11-27T17:00:45.746 INFO:teuthology.orchestra.run.smithi154.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2022-11-27T17:00:45.769 INFO:teuthology.orchestra.run.smithi154.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 304 kB/s | 23 kB 00:00 2022-11-27T17:00:45.774 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:00:45.774 INFO:teuthology.orchestra.run.smithi154.stdout:Total 22 MB/s | 28 MB 00:01 2022-11-27T17:00:45.852 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T17:00:45.891 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T17:00:45.891 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T17:00:46.205 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T17:00:46.208 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T17:00:46.684 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T17:00:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T17:00:46.906 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2022-11-27T17:00:47.108 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2022-11-27T17:00:47.150 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2022-11-27T17:00:47.311 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2022-11-27T17:00:47.397 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2022-11-27T17:00:47.457 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T17:00:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2022-11-27T17:00:47.615 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2022-11-27T17:00:47.698 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2022-11-27T17:00:47.701 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2022-11-27T17:00:47.765 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2022-11-27T17:00:47.882 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2022-11-27T17:00:47.926 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2022-11-27T17:00:47.943 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2022-11-27T17:00:48.037 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2022-11-27T17:00:48.078 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2022-11-27T17:00:48.181 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2022-11-27T17:00:48.206 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2022-11-27T17:00:48.248 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2022-11-27T17:00:48.382 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2022-11-27T17:00:48.391 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2022-11-27T17:00:48.430 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2022-11-27T17:00:48.567 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2022-11-27T17:00:48.655 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2022-11-27T17:00:48.684 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2022-11-27T17:00:48.812 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2022-11-27T17:00:48.843 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2022-11-27T17:00:48.913 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2022-11-27T17:00:49.004 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2022-11-27T17:00:49.058 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2022-11-27T17:00:49.140 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2022-11-27T17:00:49.161 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2022-11-27T17:00:49.257 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2022-11-27T17:00:49.355 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2022-11-27T17:00:49.584 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2022-11-27T17:00:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2022-11-27T17:00:49.630 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2022-11-27T17:00:49.846 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2022-11-27T17:00:49.886 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2022-11-27T17:00:49.905 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2022-11-27T17:00:50.081 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2022-11-27T17:00:50.090 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2022-11-27T17:00:50.133 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2022-11-27T17:00:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2022-11-27T17:00:50.399 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2022-11-27T17:00:50.474 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2022-11-27T17:00:50.595 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T17:00:50.634 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T17:00:50.762 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2022-11-27T17:00:50.878 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2022-11-27T17:00:50.917 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2022-11-27T17:00:50.972 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2022-11-27T17:00:51.092 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2022-11-27T17:00:51.094 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2022-11-27T17:00:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2022-11-27T17:00:51.319 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2022-11-27T17:00:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2022-11-27T17:00:51.499 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2022-11-27T17:00:51.535 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T17:00:51.539 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2022-11-27T17:00:51.819 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2022-11-27T17:00:51.859 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2022-11-27T17:00:51.879 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2022-11-27T17:00:52.056 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2022-11-27T17:00:52.064 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2022-11-27T17:00:52.145 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2022-11-27T17:00:52.260 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2022-11-27T17:00:52.322 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2022-11-27T17:00:52.382 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2022-11-27T17:00:52.464 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2022-11-27T17:00:52.695 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2022-11-27T17:00:52.722 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2022-11-27T17:00:52.772 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2022-11-27T17:00:53.005 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2022-11-27T17:00:53.012 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2022-11-27T17:00:53.052 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2022-11-27T17:00:53.261 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2022-11-27T17:00:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 27/32 2022-11-27T17:00:53.417 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 27/32 2022-11-27T17:00:53.520 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T17:00:53.606 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T17:00:53.623 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2022-11-27T17:00:54.011 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2022-11-27T17:00:54.361 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 27/32 2022-11-27T17:00:54.532 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T17:00:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T17:00:54.970 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T17:00:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T17:00:55.167 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T17:00:55.426 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T17:00:55.541 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T17:00:55.619 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T17:00:55.725 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T17:00:55.801 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T17:00:55.898 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T17:00:55.907 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T17:00:55.935 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T17:00:55.953 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T17:00:56.083 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T17:00:56.084 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T17:00:56.092 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T17:00:56.100 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T17:00:56.214 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T17:00:56.456 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T17:00:56.482 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T17:00:56.482 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:15.2.17-0.el8.x86_64 1/32 2022-11-27T17:00:56.482 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:15.2.17-0.el8.x86_64 2/32 2022-11-27T17:00:56.483 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:15.2.17-0.el8.x86_64 3/32 2022-11-27T17:00:56.483 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:15.2.17-0.el8.x86_64 4/32 2022-11-27T17:00:56.483 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:15.2.17-0.el8.x86_64 5/32 2022-11-27T17:00:56.483 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 6/32 2022-11-27T17:00:56.484 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2022-11-27T17:00:56.484 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2022-11-27T17:00:56.484 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2022-11-27T17:00:56.484 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2022-11-27T17:00:56.484 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2022-11-27T17:00:56.485 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2022-11-27T17:00:56.485 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2022-11-27T17:00:56.485 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2022-11-27T17:00:56.485 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2022-11-27T17:00:56.486 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2022-11-27T17:00:56.486 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T17:00:56.486 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2022-11-27T17:00:56.486 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2022-11-27T17:00:56.486 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2022-11-27T17:00:56.487 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2022-11-27T17:00:56.487 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2022-11-27T17:00:56.487 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2022-11-27T17:00:56.487 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2022-11-27T17:00:56.488 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2022-11-27T17:00:56.488 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2022-11-27T17:00:56.488 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2022-11-27T17:00:56.488 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2022-11-27T17:00:56.488 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2022-11-27T17:00:56.489 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2022-11-27T17:00:56.489 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2022-11-27T17:00:56.633 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T17:00:56.634 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:15.2.17-0.el8.x86_64 1/32 2022-11-27T17:00:56.634 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:15.2.17-0.el8.x86_64 2/32 2022-11-27T17:00:56.634 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:15.2.17-0.el8.x86_64 3/32 2022-11-27T17:00:56.634 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:15.2.17-0.el8.x86_64 4/32 2022-11-27T17:00:56.635 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:15.2.17-0.el8.x86_64 5/32 2022-11-27T17:00:56.635 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 6/32 2022-11-27T17:00:56.635 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2022-11-27T17:00:56.636 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2022-11-27T17:00:56.636 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2022-11-27T17:00:56.636 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2022-11-27T17:00:56.636 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2022-11-27T17:00:56.637 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2022-11-27T17:00:56.637 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2022-11-27T17:00:56.637 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2022-11-27T17:00:56.637 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2022-11-27T17:00:56.638 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2022-11-27T17:00:56.638 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T17:00:56.638 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2022-11-27T17:00:56.638 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2022-11-27T17:00:56.639 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2022-11-27T17:00:56.639 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2022-11-27T17:00:56.639 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2022-11-27T17:00:56.639 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2022-11-27T17:00:56.640 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2022-11-27T17:00:56.640 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2022-11-27T17:00:56.640 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2022-11-27T17:00:56.640 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2022-11-27T17:00:56.641 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2022-11-27T17:00:56.641 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2022-11-27T17:00:56.641 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2022-11-27T17:00:56.641 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2022-11-27T17:00:56.658 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T17:00:56.842 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T17:00:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T17:00:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T17:00:57.148 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T17:00:57.277 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2022-11-27T17:00:57.277 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:57.278 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2022-11-27T17:00:57.279 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:57.279 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:57.279 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:57.279 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 2022-11-27T17:00:57.280 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:57.280 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:57.280 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T17:00:57.280 INFO:teuthology.orchestra.run.smithi165.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T17:00:57.281 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T17:00:57.281 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T17:00:57.281 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T17:00:57.281 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T17:00:57.282 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T17:00:57.282 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T17:00:57.282 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T17:00:57.282 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T17:00:57.282 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T17:00:57.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T17:00:57.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T17:00:57.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T17:00:57.283 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T17:00:57.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T17:00:57.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T17:00:57.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T17:00:57.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T17:00:57.284 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T17:00:57.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T17:00:57.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T17:00:57.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T17:00:57.285 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T17:00:57.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T17:00:57.286 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T17:00:57.286 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:00:57.286 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:00:57.519 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-base 2022-11-27T17:00:57.530 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T17:00:57.531 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:15.2.17-0.el8.x86_64 1/32 2022-11-27T17:00:57.531 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:15.2.17-0.el8.x86_64 2/32 2022-11-27T17:00:57.531 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:15.2.17-0.el8.x86_64 3/32 2022-11-27T17:00:57.532 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:15.2.17-0.el8.x86_64 4/32 2022-11-27T17:00:57.532 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:15.2.17-0.el8.x86_64 5/32 2022-11-27T17:00:57.533 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 6/32 2022-11-27T17:00:57.533 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2022-11-27T17:00:57.534 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2022-11-27T17:00:57.534 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2022-11-27T17:00:57.534 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2022-11-27T17:00:57.535 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2022-11-27T17:00:57.535 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2022-11-27T17:00:57.535 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2022-11-27T17:00:57.535 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2022-11-27T17:00:57.535 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2022-11-27T17:00:57.536 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2022-11-27T17:00:57.536 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T17:00:57.536 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2022-11-27T17:00:57.536 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2022-11-27T17:00:57.537 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2022-11-27T17:00:57.537 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2022-11-27T17:00:57.537 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2022-11-27T17:00:57.537 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2022-11-27T17:00:57.537 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2022-11-27T17:00:57.538 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2022-11-27T17:00:57.538 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2022-11-27T17:00:57.538 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2022-11-27T17:00:57.538 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2022-11-27T17:00:57.539 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2022-11-27T17:00:57.539 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2022-11-27T17:00:57.540 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2022-11-27T17:00:57.543 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2022-11-27T17:00:57.543 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:57.543 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2022-11-27T17:00:57.544 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:57.544 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:57.544 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:57.545 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 2022-11-27T17:00:57.545 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:57.545 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:57.546 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T17:00:57.546 INFO:teuthology.orchestra.run.smithi159.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T17:00:57.547 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T17:00:57.547 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T17:00:57.548 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T17:00:57.548 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T17:00:57.549 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T17:00:57.549 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T17:00:57.550 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T17:00:57.550 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T17:00:57.550 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T17:00:57.551 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T17:00:57.551 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T17:00:57.552 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T17:00:57.552 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T17:00:57.553 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T17:00:57.553 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T17:00:57.554 INFO:teuthology.orchestra.run.smithi159.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T17:00:57.554 INFO:teuthology.orchestra.run.smithi159.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T17:00:57.554 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T17:00:57.555 INFO:teuthology.orchestra.run.smithi159.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T17:00:57.555 INFO:teuthology.orchestra.run.smithi159.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T17:00:57.556 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T17:00:57.556 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T17:00:57.557 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T17:00:57.557 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T17:00:57.557 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:00:57.558 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:00:57.800 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-base 2022-11-27T17:00:58.125 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:03 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:00:58.406 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:03 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:00:58.447 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2022-11-27T17:00:58.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:58.448 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2022-11-27T17:00:58.448 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:58.448 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:58.448 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:58.449 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 2022-11-27T17:00:58.449 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:58.449 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:15.2.17-0.el8.x86_64 2022-11-27T17:00:58.449 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T17:00:58.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T17:00:58.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T17:00:58.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T17:00:58.451 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T17:00:58.451 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T17:00:58.451 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T17:00:58.452 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T17:00:58.452 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T17:00:58.452 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T17:00:58.452 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T17:00:58.453 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T17:00:58.453 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T17:00:58.453 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T17:00:58.453 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T17:00:58.454 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T17:00:58.454 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T17:00:58.454 INFO:teuthology.orchestra.run.smithi154.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T17:00:58.454 INFO:teuthology.orchestra.run.smithi154.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T17:00:58.455 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T17:00:58.455 INFO:teuthology.orchestra.run.smithi154.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T17:00:58.455 INFO:teuthology.orchestra.run.smithi154.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T17:00:58.456 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T17:00:58.456 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T17:00:58.456 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T17:00:58.456 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T17:00:58.456 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:00:58.457 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:00:58.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-base 2022-11-27T17:00:59.122 INFO:teuthology.orchestra.run.smithi165.stdout:Package ceph-base-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:00:59.213 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:00:59.214 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T17:00:59.214 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:00:59.286 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:04 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:00:59.291 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-immutable-object-cache 2022-11-27T17:00:59.410 INFO:teuthology.orchestra.run.smithi159.stdout:Package ceph-base-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:00:59.501 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:00:59.502 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T17:00:59.502 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:00:59.589 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-immutable-object-cache 2022-11-27T17:00:59.895 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:04 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:00.198 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:05 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:00.295 INFO:teuthology.orchestra.run.smithi154.stdout:Package ceph-base-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:00.385 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:00.387 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T17:01:00.387 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:00.491 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-immutable-object-cache 2022-11-27T17:01:00.987 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:00.988 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2022-11-27T17:01:00.988 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:00.989 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2022-11-27T17:01:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:15.2.17-0.el8 ceph 203 k 2022-11-27T17:01:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:00.990 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T17:01:00.990 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:00.990 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2022-11-27T17:01:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:00.991 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 203 k 2022-11-27T17:01:00.991 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 599 k 2022-11-27T17:01:00.992 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T17:01:01.056 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-immutable-object-cache-15.2.17-0.el8.x86_6 3.0 MB/s | 203 kB 00:00 2022-11-27T17:01:01.125 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:06 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:01.126 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:01.127 INFO:teuthology.orchestra.run.smithi165.stdout:Total 2.9 MB/s | 203 kB 00:00 2022-11-27T17:01:01.127 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T17:01:01.127 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T17:01:01.127 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T17:01:01.128 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T17:01:01.128 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T17:01:01.240 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T17:01:01.280 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:01.281 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:01.282 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repo Size 2022-11-27T17:01:01.282 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:01.282 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2022-11-27T17:01:01.283 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:15.2.17-0.el8 ceph 203 k 2022-11-27T17:01:01.283 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:01.283 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T17:01:01.284 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:01.284 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2022-11-27T17:01:01.284 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:01.285 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 203 k 2022-11-27T17:01:01.285 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 599 k 2022-11-27T17:01:01.286 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T17:01:01.352 INFO:teuthology.orchestra.run.smithi159.stdout:ceph-immutable-object-cache-15.2.17-0.el8.x86_6 2.8 MB/s | 203 kB 00:00 2022-11-27T17:01:01.353 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:01.354 INFO:teuthology.orchestra.run.smithi159.stdout:Total 2.8 MB/s | 203 kB 00:00 2022-11-27T17:01:01.355 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T17:01:01.364 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T17:01:01.365 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T17:01:01.413 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T17:01:01.415 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T17:01:01.426 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:01.531 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T17:01:01.732 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:02.111 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:02.205 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:02.206 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:02.206 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repo Size 2022-11-27T17:01:02.206 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:02.207 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2022-11-27T17:01:02.207 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:15.2.17-0.el8 ceph 203 k 2022-11-27T17:01:02.207 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:02.207 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T17:01:02.208 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:02.208 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2022-11-27T17:01:02.208 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:02.208 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 203 k 2022-11-27T17:01:02.209 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 599 k 2022-11-27T17:01:02.209 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T17:01:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:02.232 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2022-11-27T17:01:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:02.233 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:02.249 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-immutable-object-cache-15.2.17-0.el8.x86_6 4.7 MB/s | 203 kB 00:00 2022-11-27T17:01:02.251 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:02.251 INFO:teuthology.orchestra.run.smithi154.stdout:Total 4.6 MB/s | 203 kB 00:00 2022-11-27T17:01:02.251 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T17:01:02.261 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T17:01:02.261 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T17:01:02.309 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T17:01:02.312 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T17:01:02.408 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr 2022-11-27T17:01:02.446 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T17:01:02.514 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:02.514 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:02.515 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2022-11-27T17:01:02.516 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:02.516 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:02.517 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:02.641 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:02.720 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-mgr 2022-11-27T17:01:03.021 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:03.029 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:08 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:03.338 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:08 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:03.479 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:03.480 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:03.480 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2022-11-27T17:01:03.480 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:03.480 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:03.481 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:03.676 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-mgr 2022-11-27T17:01:04.025 INFO:teuthology.orchestra.run.smithi165.stdout:Package ceph-mgr-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:04.116 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:04.117 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T17:01:04.118 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:04.208 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-fuse 2022-11-27T17:01:04.293 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:09 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:04.341 INFO:teuthology.orchestra.run.smithi159.stdout:Package ceph-mgr-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:04.431 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:04.432 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T17:01:04.432 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:04.530 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-fuse 2022-11-27T17:01:04.806 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:09 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:05.141 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:10 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:05.287 INFO:teuthology.orchestra.run.smithi154.stdout:Package ceph-mgr-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:05.376 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:05.377 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T17:01:05.377 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:05.463 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-fuse 2022-11-27T17:01:05.893 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:05.895 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:05.895 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:05.895 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2022-11-27T17:01:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:15.2.17-0.el8 ceph 572 k 2022-11-27T17:01:05.896 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2022-11-27T17:01:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2022-11-27T17:01:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:05.897 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T17:01:05.897 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:05.897 INFO:teuthology.orchestra.run.smithi165.stdout:Install 2 Packages 2022-11-27T17:01:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:05.897 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 655 k 2022-11-27T17:01:05.898 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 1.8 M 2022-11-27T17:01:05.898 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T17:01:05.989 INFO:teuthology.orchestra.run.smithi165.stdout:(1/2): ceph-fuse-15.2.17-0.el8.x86_64.rpm 6.0 MB/s | 572 kB 00:00 2022-11-27T17:01:06.025 INFO:teuthology.orchestra.run.smithi165.stdout:(2/2): fuse-2.9.7-16.el8.x86_64.rpm 642 kB/s | 83 kB 00:00 2022-11-27T17:01:06.026 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:06.027 INFO:teuthology.orchestra.run.smithi165.stdout:Total 4.9 MB/s | 655 kB 00:00 2022-11-27T17:01:06.037 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T17:01:06.057 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T17:01:06.057 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T17:01:06.071 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:11 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:06.126 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T17:01:06.130 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T17:01:06.222 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:06.222 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:06.223 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:06.223 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:06.223 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2022-11-27T17:01:06.223 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:15.2.17-0.el8 ceph 572 k 2022-11-27T17:01:06.224 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2022-11-27T17:01:06.224 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2022-11-27T17:01:06.224 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:06.225 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T17:01:06.225 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:06.225 INFO:teuthology.orchestra.run.smithi159.stdout:Install 2 Packages 2022-11-27T17:01:06.225 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:06.226 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 655 k 2022-11-27T17:01:06.226 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 1.8 M 2022-11-27T17:01:06.227 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T17:01:06.281 INFO:teuthology.orchestra.run.smithi159.stdout:(1/2): ceph-fuse-15.2.17-0.el8.x86_64.rpm 9.8 MB/s | 572 kB 00:00 2022-11-27T17:01:06.307 INFO:teuthology.orchestra.run.smithi159.stdout:(2/2): fuse-2.9.7-16.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2022-11-27T17:01:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T17:01:06.308 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:06.308 INFO:teuthology.orchestra.run.smithi159.stdout:Total 7.6 MB/s | 655 kB 00:00 2022-11-27T17:01:06.317 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T17:01:06.338 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T17:01:06.338 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T17:01:06.406 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T17:01:06.409 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T17:01:06.521 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2022-11-27T17:01:06.616 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T17:01:06.715 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T17:01:06.861 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2022-11-27T17:01:07.021 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T17:01:07.055 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T17:01:07.055 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:15.2.17-0.el8.x86_64 1/2 2022-11-27T17:01:07.148 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:07.150 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:07.150 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:07.150 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:07.150 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2022-11-27T17:01:07.151 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:15.2.17-0.el8 ceph 572 k 2022-11-27T17:01:07.151 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2022-11-27T17:01:07.151 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2022-11-27T17:01:07.152 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:07.152 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T17:01:07.152 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:07.152 INFO:teuthology.orchestra.run.smithi154.stdout:Install 2 Packages 2022-11-27T17:01:07.153 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:07.153 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 655 k 2022-11-27T17:01:07.154 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 1.8 M 2022-11-27T17:01:07.154 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T17:01:07.228 INFO:teuthology.orchestra.run.smithi154.stdout:(1/2): ceph-fuse-15.2.17-0.el8.x86_64.rpm 7.2 MB/s | 572 kB 00:00 2022-11-27T17:01:07.245 INFO:teuthology.orchestra.run.smithi154.stdout:(2/2): fuse-2.9.7-16.el8.x86_64.rpm 883 kB/s | 83 kB 00:00 2022-11-27T17:01:07.246 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:07.246 INFO:teuthology.orchestra.run.smithi154.stdout:Total 6.7 MB/s | 655 kB 00:00 2022-11-27T17:01:07.256 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T17:01:07.276 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T17:01:07.277 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T17:01:07.348 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T17:01:07.351 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T17:01:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T17:01:07.396 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:15.2.17-0.el8.x86_64 1/2 2022-11-27T17:01:07.448 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2022-11-27T17:01:07.448 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:07.448 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2022-11-27T17:01:07.449 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:15.2.17-0.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2022-11-27T17:01:07.449 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:07.449 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:07.522 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T17:01:07.651 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librados-devel 2022-11-27T17:01:07.786 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2022-11-27T17:01:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2022-11-27T17:01:07.838 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:07.838 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2022-11-27T17:01:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:15.2.17-0.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2022-11-27T17:01:07.839 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:07.840 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:07.988 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T17:01:08.091 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install librados-devel 2022-11-27T17:01:08.264 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:13 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:08.346 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T17:01:08.347 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:15.2.17-0.el8.x86_64 1/2 2022-11-27T17:01:08.707 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:13 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2022-11-27T17:01:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:08.756 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2022-11-27T17:01:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:15.2.17-0.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2022-11-27T17:01:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:08.757 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:08.947 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install librados-devel 2022-11-27T17:01:09.358 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:09.359 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:09.360 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:09.360 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:09.360 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2022-11-27T17:01:09.360 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T17:01:09.361 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:09.361 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T17:01:09.361 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:09.361 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2022-11-27T17:01:09.361 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:09.362 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 140 k 2022-11-27T17:01:09.362 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 478 k 2022-11-27T17:01:09.362 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T17:01:09.411 INFO:teuthology.orchestra.run.smithi165.stdout:librados-devel-15.2.17-0.el8.x86_64.rpm 2.7 MB/s | 140 kB 00:00 2022-11-27T17:01:09.412 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:09.412 INFO:teuthology.orchestra.run.smithi165.stdout:Total 2.6 MB/s | 140 kB 00:00 2022-11-27T17:01:09.412 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T17:01:09.417 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T17:01:09.417 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T17:01:09.463 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T17:01:09.465 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T17:01:09.560 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:14 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T17:01:09.738 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:09.783 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:09.784 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:09.784 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:09.785 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:09.785 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2022-11-27T17:01:09.786 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T17:01:09.786 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:09.786 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T17:01:09.787 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:09.787 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2022-11-27T17:01:09.787 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:09.788 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 140 k 2022-11-27T17:01:09.788 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 478 k 2022-11-27T17:01:09.788 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T17:01:09.836 INFO:teuthology.orchestra.run.smithi159.stdout:librados-devel-15.2.17-0.el8.x86_64.rpm 2.7 MB/s | 140 kB 00:00 2022-11-27T17:01:09.837 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:09.837 INFO:teuthology.orchestra.run.smithi159.stdout:Total 2.6 MB/s | 140 kB 00:00 2022-11-27T17:01:09.838 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T17:01:09.842 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T17:01:09.842 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T17:01:09.888 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T17:01:09.890 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T17:01:09.999 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T17:01:10.035 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:10.456 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:10.514 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:10.514 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:10.514 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2022-11-27T17:01:10.515 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:10.515 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:10.515 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:10.644 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:10.645 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:10.645 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:10.646 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:10.646 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2022-11-27T17:01:10.646 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T17:01:10.646 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:10.647 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T17:01:10.647 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:10.647 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2022-11-27T17:01:10.647 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:10.648 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 140 k 2022-11-27T17:01:10.648 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 478 k 2022-11-27T17:01:10.648 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T17:01:10.680 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install libcephfs2 2022-11-27T17:01:10.699 INFO:teuthology.orchestra.run.smithi154.stdout:librados-devel-15.2.17-0.el8.x86_64.rpm 2.6 MB/s | 140 kB 00:00 2022-11-27T17:01:10.700 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:10.701 INFO:teuthology.orchestra.run.smithi154.stdout:Total 2.5 MB/s | 140 kB 00:00 2022-11-27T17:01:10.701 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T17:01:10.705 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T17:01:10.706 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T17:01:10.751 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T17:01:10.754 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T17:01:10.856 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T17:01:11.028 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:11.291 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:16 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:11.303 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:11.336 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:11.336 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:11.336 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2022-11-27T17:01:11.337 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:11.337 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:11.337 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:12.298 INFO:teuthology.orchestra.run.smithi165.stdout:Package libcephfs2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:12.390 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:12.391 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T17:01:12.392 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:12.445 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:12.445 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:12.445 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2022-11-27T17:01:12.446 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:12.446 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:12.446 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:12.482 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install libcephfs-devel 2022-11-27T17:01:12.666 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install libcephfs2 2022-11-27T17:01:13.088 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:18 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:13.225 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install libcephfs2 2022-11-27T17:01:13.278 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:18 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:13.840 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:18 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:14.180 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:14.181 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:14.181 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:14.181 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:14.182 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2022-11-27T17:01:14.182 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:15.2.17-0.el8 ceph 22 k 2022-11-27T17:01:14.183 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:14.183 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T17:01:14.183 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:14.184 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2022-11-27T17:01:14.184 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:14.185 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 22 k 2022-11-27T17:01:14.185 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 78 k 2022-11-27T17:01:14.185 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T17:01:14.255 INFO:teuthology.orchestra.run.smithi165.stdout:libcephfs-devel-15.2.17-0.el8.x86_64.rpm 298 kB/s | 22 kB 00:00 2022-11-27T17:01:14.256 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:14.256 INFO:teuthology.orchestra.run.smithi165.stdout:Total 292 kB/s | 22 kB 00:00 2022-11-27T17:01:14.256 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T17:01:14.259 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T17:01:14.259 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T17:01:14.261 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T17:01:14.264 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T17:01:14.268 INFO:teuthology.orchestra.run.smithi159.stdout:Package libcephfs2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T17:01:14.359 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:14.360 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T17:01:14.360 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:14.448 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install libcephfs-devel 2022-11-27T17:01:14.453 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:14.679 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:14.843 INFO:teuthology.orchestra.run.smithi154.stdout:Package libcephfs2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:14.935 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:14.936 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T17:01:14.936 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:15.013 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install libcephfs-devel 2022-11-27T17:01:15.046 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:20 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:15.064 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:15.064 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:15.064 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2022-11-27T17:01:15.064 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:15.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:15.065 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:15.200 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librados2 2022-11-27T17:01:15.612 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:20 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:15.808 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:20 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:16.129 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:16.130 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:16.131 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:16.131 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:16.131 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2022-11-27T17:01:16.132 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:15.2.17-0.el8 ceph 22 k 2022-11-27T17:01:16.132 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:16.132 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T17:01:16.133 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:16.133 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2022-11-27T17:01:16.134 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:16.134 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 22 k 2022-11-27T17:01:16.134 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 78 k 2022-11-27T17:01:16.135 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T17:01:16.174 INFO:teuthology.orchestra.run.smithi159.stdout:libcephfs-devel-15.2.17-0.el8.x86_64.rpm 513 kB/s | 22 kB 00:00 2022-11-27T17:01:16.175 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:16.176 INFO:teuthology.orchestra.run.smithi159.stdout:Total 497 kB/s | 22 kB 00:00 2022-11-27T17:01:16.176 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T17:01:16.178 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T17:01:16.178 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T17:01:16.179 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T17:01:16.182 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T17:01:16.233 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T17:01:16.402 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:16.629 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:16.683 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:16.684 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:16.684 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:16.685 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:16.685 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2022-11-27T17:01:16.685 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:15.2.17-0.el8 ceph 22 k 2022-11-27T17:01:16.685 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:16.686 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T17:01:16.686 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:16.686 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2022-11-27T17:01:16.687 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:16.687 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 22 k 2022-11-27T17:01:16.687 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 78 k 2022-11-27T17:01:16.688 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T17:01:16.755 INFO:teuthology.orchestra.run.smithi154.stdout:libcephfs-devel-15.2.17-0.el8.x86_64.rpm 312 kB/s | 22 kB 00:00 2022-11-27T17:01:16.756 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:16.756 INFO:teuthology.orchestra.run.smithi154.stdout:Total 306 kB/s | 22 kB 00:00 2022-11-27T17:01:16.756 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T17:01:16.759 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T17:01:16.759 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T17:01:16.760 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T17:01:16.763 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T17:01:16.798 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T17:01:16.889 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:16.890 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T17:01:16.891 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:16.981 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librbd1 2022-11-27T17:01:17.006 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:17.006 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:17.007 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2022-11-27T17:01:17.007 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:17.007 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:17.007 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:17.027 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:17.126 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install librados2 2022-11-27T17:01:17.252 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:17.585 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:22 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:17.650 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:17.651 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:17.651 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2022-11-27T17:01:17.651 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:17.652 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:17.652 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:17.729 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:22 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:17.782 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install librados2 2022-11-27T17:01:18.384 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:23 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:18.584 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:18.674 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:18.686 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T17:01:18.686 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:18.715 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:18.767 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rados 2022-11-27T17:01:18.807 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:18.808 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T17:01:18.808 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:18.890 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install librbd1 2022-11-27T17:01:19.367 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:24 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:19.385 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:19.475 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:19.476 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T17:01:19.477 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:19.488 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:24 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:19.564 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install librbd1 2022-11-27T17:01:20.169 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:25 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:20.361 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rados-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:20.453 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:20.454 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T17:01:20.454 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:20.489 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:20.536 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rgw 2022-11-27T17:01:20.581 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:20.582 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T17:01:20.582 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:20.680 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-rados 2022-11-27T17:01:21.147 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:26 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:21.156 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:21.247 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:21.248 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T17:01:21.248 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:21.277 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:26 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:21.333 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install python3-rados 2022-11-27T17:01:21.934 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:26 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:22.142 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rgw-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:22.234 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:22.235 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T17:01:22.235 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:22.282 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-rados-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:22.324 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-cephfs 2022-11-27T17:01:22.373 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:22.374 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T17:01:22.374 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:22.463 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-rgw 2022-11-27T17:01:22.925 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:27 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:22.925 INFO:teuthology.orchestra.run.smithi154.stdout:Package python3-rados-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:23.019 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:23.020 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T17:01:23.021 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:23.063 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:28 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:23.131 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install python3-rgw 2022-11-27T17:01:23.740 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:28 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:23.908 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-cephfs-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:23.999 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:24.000 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T17:01:24.000 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:24.054 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-rgw-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:24.086 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rbd 2022-11-27T17:01:24.144 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:24.145 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T17:01:24.145 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:24.238 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-cephfs 2022-11-27T17:01:24.689 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:29 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:24.728 INFO:teuthology.orchestra.run.smithi154.stdout:Package python3-rgw-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:24.820 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:24.821 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T17:01:24.821 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:24.837 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:29 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:24.914 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install python3-cephfs 2022-11-27T17:01:25.558 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:30 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:25.673 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rbd-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:25.764 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:25.766 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T17:01:25.766 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:25.826 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-cephfs-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:25.857 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-fuse 2022-11-27T17:01:25.918 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:25.919 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T17:01:25.919 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:26.030 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install python3-rbd 2022-11-27T17:01:26.461 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:31 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:26.536 INFO:teuthology.orchestra.run.smithi154.stdout:Package python3-cephfs-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:26.627 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:26.628 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:31 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:26.628 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T17:01:26.629 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:26.733 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install python3-rbd 2022-11-27T17:01:27.333 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:32 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:27.540 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:27.541 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:27.542 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:27.542 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2022-11-27T17:01:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:15.2.17-0.el8 ceph 73 k 2022-11-27T17:01:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:27.543 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T17:01:27.544 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:27.544 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2022-11-27T17:01:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:27.545 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 73 k 2022-11-27T17:01:27.545 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 188 k 2022-11-27T17:01:27.545 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T17:01:27.610 INFO:teuthology.orchestra.run.smithi159.stdout:Package python3-rbd-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:27.628 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-fuse-15.2.17-0.el8.x86_64.rpm 843 kB/s | 73 kB 00:00 2022-11-27T17:01:27.629 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:27.630 INFO:teuthology.orchestra.run.smithi165.stdout:Total 830 kB/s | 73 kB 00:00 2022-11-27T17:01:27.630 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T17:01:27.639 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T17:01:27.640 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T17:01:27.685 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T17:01:27.688 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T17:01:27.702 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:27.703 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T17:01:27.703 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:27.816 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install rbd-fuse 2022-11-27T17:01:28.313 INFO:teuthology.orchestra.run.smithi154.stdout:Package python3-rbd-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:28.403 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:28.404 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T17:01:28.404 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:28.406 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T17:01:28.420 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:33 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:28.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install rbd-fuse 2022-11-27T17:01:28.618 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:28.877 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:29.089 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:34 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:29.346 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:29.347 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2022-11-27T17:01:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:29.347 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:29.347 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:29.503 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:29.504 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:29.504 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:29.505 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:29.505 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2022-11-27T17:01:29.505 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:15.2.17-0.el8 ceph 73 k 2022-11-27T17:01:29.505 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:29.506 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T17:01:29.506 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:29.506 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2022-11-27T17:01:29.506 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:29.507 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 73 k 2022-11-27T17:01:29.507 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 188 k 2022-11-27T17:01:29.507 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T17:01:29.536 INFO:teuthology.orchestra.run.smithi159.stdout:rbd-fuse-15.2.17-0.el8.x86_64.rpm 2.2 MB/s | 73 kB 00:00 2022-11-27T17:01:29.537 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:29.538 INFO:teuthology.orchestra.run.smithi159.stdout:Total 2.1 MB/s | 73 kB 00:00 2022-11-27T17:01:29.538 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T17:01:29.547 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T17:01:29.547 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T17:01:29.586 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-mirror 2022-11-27T17:01:29.592 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T17:01:29.595 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T17:01:29.696 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T17:01:29.905 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:30.159 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:30.188 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:30.189 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:30.190 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:30.190 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:30.190 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2022-11-27T17:01:30.191 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:15.2.17-0.el8 ceph 73 k 2022-11-27T17:01:30.191 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:30.191 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T17:01:30.191 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:30.192 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2022-11-27T17:01:30.192 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:30.192 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 73 k 2022-11-27T17:01:30.192 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 188 k 2022-11-27T17:01:30.193 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T17:01:30.199 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:35 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:30.250 INFO:teuthology.orchestra.run.smithi154.stdout:rbd-fuse-15.2.17-0.el8.x86_64.rpm 1.2 MB/s | 73 kB 00:00 2022-11-27T17:01:30.251 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:30.251 INFO:teuthology.orchestra.run.smithi154.stdout:Total 1.2 MB/s | 73 kB 00:00 2022-11-27T17:01:30.251 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T17:01:30.260 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T17:01:30.260 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T17:01:30.306 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T17:01:30.309 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T17:01:30.426 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T17:01:30.616 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:30.646 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:30.647 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:30.648 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2022-11-27T17:01:30.648 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:30.649 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:30.649 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:30.860 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install rbd-mirror 2022-11-27T17:01:30.866 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:31.280 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:31.281 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:31.281 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:31.281 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:31.282 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2022-11-27T17:01:31.282 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:15.2.17-0.el8 ceph 2.7 M 2022-11-27T17:01:31.282 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:31.282 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T17:01:31.282 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:31.283 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2022-11-27T17:01:31.283 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:31.283 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 2.7 M 2022-11-27T17:01:31.283 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 9.8 M 2022-11-27T17:01:31.284 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T17:01:31.309 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:31.309 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:31.309 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2022-11-27T17:01:31.310 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:31.310 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:31.310 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:31.369 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-mirror-15.2.17-0.el8.x86_64.rpm 31 MB/s | 2.7 MB 00:00 2022-11-27T17:01:31.371 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:31.371 INFO:teuthology.orchestra.run.smithi165.stdout:Total 31 MB/s | 2.7 MB 00:00 2022-11-27T17:01:31.371 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T17:01:31.381 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T17:01:31.382 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T17:01:31.439 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T17:01:31.442 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T17:01:31.469 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:36 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:31.507 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install rbd-mirror 2022-11-27T17:01:31.784 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T17:01:31.979 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:32.118 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:37 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:32.359 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:32.556 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:32.557 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:32.557 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:32.557 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:32.557 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2022-11-27T17:01:32.558 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:15.2.17-0.el8 ceph 2.7 M 2022-11-27T17:01:32.558 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:32.558 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T17:01:32.559 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:32.559 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2022-11-27T17:01:32.559 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:32.559 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 2.7 M 2022-11-27T17:01:32.560 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 9.8 M 2022-11-27T17:01:32.560 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T17:01:32.640 INFO:teuthology.orchestra.run.smithi159.stdout:rbd-mirror-15.2.17-0.el8.x86_64.rpm 33 MB/s | 2.7 MB 00:00 2022-11-27T17:01:32.641 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:32.641 INFO:teuthology.orchestra.run.smithi159.stdout:Total 33 MB/s | 2.7 MB 00:00 2022-11-27T17:01:32.641 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T17:01:32.651 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T17:01:32.652 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T17:01:32.707 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T17:01:32.710 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T17:01:32.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:32.786 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:32.786 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2022-11-27T17:01:32.786 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:32.787 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:32.787 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:32.984 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-nbd 2022-11-27T17:01:33.051 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T17:01:33.199 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:33.201 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:33.201 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:33.201 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:33.201 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2022-11-27T17:01:33.202 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:15.2.17-0.el8 ceph 2.7 M 2022-11-27T17:01:33.202 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:33.202 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T17:01:33.202 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:33.202 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2022-11-27T17:01:33.203 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:33.203 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 2.7 M 2022-11-27T17:01:33.203 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 9.8 M 2022-11-27T17:01:33.203 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T17:01:33.286 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:33.319 INFO:teuthology.orchestra.run.smithi154.stdout:rbd-mirror-15.2.17-0.el8.x86_64.rpm 23 MB/s | 2.7 MB 00:00 2022-11-27T17:01:33.320 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:33.321 INFO:teuthology.orchestra.run.smithi154.stdout:Total 23 MB/s | 2.7 MB 00:00 2022-11-27T17:01:33.321 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T17:01:33.331 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T17:01:33.332 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T17:01:33.387 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T17:01:33.390 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T17:01:33.599 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:38 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:33.653 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:33.731 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T17:01:33.952 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:34.074 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:34.074 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:34.074 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2022-11-27T17:01:34.075 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:34.075 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:34.075 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:34.273 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install rbd-nbd 2022-11-27T17:01:34.377 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:34.681 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:34.683 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:34.683 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:34.683 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2022-11-27T17:01:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T17:01:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:34.684 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2022-11-27T17:01:34.684 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2022-11-27T17:01:34.685 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2022-11-27T17:01:34.685 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:34.685 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 140 k 2022-11-27T17:01:34.685 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 418 k 2022-11-27T17:01:34.686 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2022-11-27T17:01:34.755 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-nbd-15.2.17-0.el8.x86_64.rpm 1.9 MB/s | 140 kB 00:00 2022-11-27T17:01:34.756 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:34.756 INFO:teuthology.orchestra.run.smithi165.stdout:Total 1.9 MB/s | 140 kB 00:00 2022-11-27T17:01:34.756 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2022-11-27T17:01:34.765 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2022-11-27T17:01:34.765 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2022-11-27T17:01:34.810 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2022-11-27T17:01:34.813 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2022-11-27T17:01:34.824 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:34.824 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:34.824 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2022-11-27T17:01:34.825 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:34.825 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:34.825 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:34.882 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:39 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:34.916 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2022-11-27T17:01:34.985 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install rbd-nbd 2022-11-27T17:01:35.117 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:35.351 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:35.593 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:40 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:35.817 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:35.817 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:35.817 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2022-11-27T17:01:35.818 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:35.818 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:35.818 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:35.965 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:35.967 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:35.967 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:35.967 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:35.968 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2022-11-27T17:01:35.968 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T17:01:35.968 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:35.968 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2022-11-27T17:01:35.969 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2022-11-27T17:01:35.969 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2022-11-27T17:01:35.969 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:35.970 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 140 k 2022-11-27T17:01:35.970 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 418 k 2022-11-27T17:01:35.970 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2022-11-27T17:01:36.010 INFO:teuthology.orchestra.run.smithi159.stdout:rbd-nbd-15.2.17-0.el8.x86_64.rpm 3.2 MB/s | 140 kB 00:00 2022-11-27T17:01:36.011 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:36.011 INFO:teuthology.orchestra.run.smithi159.stdout:Total 3.1 MB/s | 140 kB 00:00 2022-11-27T17:01:36.011 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2022-11-27T17:01:36.020 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2022-11-27T17:01:36.020 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2022-11-27T17:01:36.037 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librados2 2022-11-27T17:01:36.066 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2022-11-27T17:01:36.069 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2022-11-27T17:01:36.208 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2022-11-27T17:01:36.399 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:36.651 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:41 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:36.670 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:36.683 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:36.684 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:36.684 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2022-11-27T17:01:36.685 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:36.685 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2022-11-27T17:01:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T17:01:36.685 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:36.686 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2022-11-27T17:01:36.686 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2022-11-27T17:01:36.686 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2022-11-27T17:01:36.686 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:36.687 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 140 k 2022-11-27T17:01:36.687 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 418 k 2022-11-27T17:01:36.687 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2022-11-27T17:01:36.740 INFO:teuthology.orchestra.run.smithi154.stdout:rbd-nbd-15.2.17-0.el8.x86_64.rpm 2.5 MB/s | 140 kB 00:00 2022-11-27T17:01:36.741 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2022-11-27T17:01:36.741 INFO:teuthology.orchestra.run.smithi154.stdout:Total 2.5 MB/s | 140 kB 00:00 2022-11-27T17:01:36.741 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2022-11-27T17:01:36.750 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2022-11-27T17:01:36.751 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2022-11-27T17:01:36.796 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2022-11-27T17:01:36.799 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2022-11-27T17:01:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2022-11-27T17:01:37.091 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:37.129 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:37.130 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:37.130 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2022-11-27T17:01:37.131 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:37.131 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:37.132 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:37.293 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install librados2 2022-11-27T17:01:37.339 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:37.640 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:37.733 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2022-11-27T17:01:37.734 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2022-11-27T17:01:37.734 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2022-11-27T17:01:37.772 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T17:01:37.773 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:37.773 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2022-11-27T17:01:37.773 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:15.2.17-0.el8.x86_64 2022-11-27T17:01:37.774 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:37.774 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:37.828 DEBUG:teuthology.parallel:result is None 2022-11-27T17:01:37.907 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:42 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:37.948 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install librados2 2022-11-27T17:01:38.561 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:43 ago on Sun 27 Nov 2022 04:59:55 PM UTC. 2022-11-27T17:01:38.923 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:39.015 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2022-11-27T17:01:39.016 INFO:teuthology.orchestra.run.smithi159.stdout:Nothing to do. 2022-11-27T17:01:39.017 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2022-11-27T17:01:39.100 DEBUG:teuthology.parallel:result is None 2022-11-27T17:01:39.550 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T17:01:39.640 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2022-11-27T17:01:39.641 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2022-11-27T17:01:39.641 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2022-11-27T17:01:39.826 DEBUG:teuthology.parallel:result is None 2022-11-27T17:01:39.827 DEBUG:teuthology.orchestra.run.smithi154:> sudo lsb_release -is 2022-11-27T17:01:39.863 INFO:teuthology.orchestra.run.smithi154.stdout:CentOSStream 2022-11-27T17:01:39.866 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T17:01:39.867 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T17:01:39.868 INFO:teuthology.packaging:ref: None 2022-11-27T17:01:39.868 INFO:teuthology.packaging:tag: None 2022-11-27T17:01:39.869 INFO:teuthology.packaging:branch: octopus 2022-11-27T17:01:39.869 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T17:01:39.870 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T17:01:40.087 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T17:01:40.112 INFO:teuthology.orchestra.run.smithi154.stdout:15.2.17-0.el8 2022-11-27T17:01:40.113 INFO:teuthology.packaging:The installed version of ceph is 15.2.17-0.el8 2022-11-27T17:01:40.114 INFO:teuthology.task.install:The correct ceph version 15.2.17-0 is installed. 2022-11-27T17:01:40.115 DEBUG:teuthology.orchestra.run.smithi159:> sudo lsb_release -is 2022-11-27T17:01:40.154 INFO:teuthology.orchestra.run.smithi159.stdout:CentOSStream 2022-11-27T17:01:40.156 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T17:01:40.157 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T17:01:40.158 INFO:teuthology.packaging:ref: None 2022-11-27T17:01:40.158 INFO:teuthology.packaging:tag: None 2022-11-27T17:01:40.158 INFO:teuthology.packaging:branch: octopus 2022-11-27T17:01:40.159 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T17:01:40.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T17:01:40.363 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T17:01:40.386 INFO:teuthology.orchestra.run.smithi159.stdout:15.2.17-0.el8 2022-11-27T17:01:40.386 INFO:teuthology.packaging:The installed version of ceph is 15.2.17-0.el8 2022-11-27T17:01:40.387 INFO:teuthology.task.install:The correct ceph version 15.2.17-0 is installed. 2022-11-27T17:01:40.387 DEBUG:teuthology.orchestra.run.smithi165:> sudo lsb_release -is 2022-11-27T17:01:40.430 INFO:teuthology.orchestra.run.smithi165.stdout:CentOSStream 2022-11-27T17:01:40.432 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T17:01:40.432 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T17:01:40.432 INFO:teuthology.packaging:ref: None 2022-11-27T17:01:40.433 INFO:teuthology.packaging:tag: None 2022-11-27T17:01:40.433 INFO:teuthology.packaging:branch: octopus 2022-11-27T17:01:40.434 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T17:01:40.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T17:01:40.640 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T17:01:40.667 INFO:teuthology.orchestra.run.smithi165.stdout:15.2.17-0.el8 2022-11-27T17:01:40.668 INFO:teuthology.packaging:The installed version of ceph is 15.2.17-0.el8 2022-11-27T17:01:40.669 INFO:teuthology.task.install:The correct ceph version 15.2.17-0 is installed. 2022-11-27T17:01:40.669 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-11-27T17:01:40.670 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:40.671 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T17:01:40.699 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2022-11-27T17:01:40.700 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T17:01:40.731 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2022-11-27T17:01:40.731 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T17:01:40.762 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-11-27T17:01:40.763 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:40.764 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T17:01:40.792 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T17:01:40.859 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2022-11-27T17:01:40.860 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T17:01:40.888 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T17:01:40.956 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2022-11-27T17:01:40.956 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T17:01:40.987 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T17:01:41.057 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-11-27T17:01:41.058 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:41.059 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T17:01:41.087 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T17:01:41.162 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2022-11-27T17:01:41.163 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T17:01:41.190 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T17:01:41.257 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2022-11-27T17:01:41.258 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T17:01:41.286 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T17:01:41.354 INFO:teuthology.run_tasks:Running task print... 2022-11-27T17:01:41.364 INFO:teuthology.task.print:**** done installing octopus 2022-11-27T17:01:41.365 INFO:teuthology.run_tasks:Running task ceph... 2022-11-27T17:01:41.438 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-11-27T17:01:41.439 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2022-11-27T17:01:41.442 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /var/log/ceph 2022-11-27T17:01:41.444 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /var/log/ceph 2022-11-27T17:01:41.472 INFO:tasks.ceph:Disabling ceph logrotate... 2022-11-27T17:01:41.473 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T17:01:41.509 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T17:01:41.513 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T17:01:41.540 INFO:tasks.ceph:Creating extra log directories... 2022-11-27T17:01:41.541 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T17:01:41.580 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T17:01:41.583 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T17:01:41.626 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-11-27T17:01:41.626 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-11-27T17:01:41.627 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088680', 'branch': 'main', 'description': 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '7088680', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'install ceph/octopus latest\n'}], 'name': 'yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'max_mds': 2}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['client.0'], ['client.1']], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'targets': {'smithi154.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRwDQ8CUvWoPdXlOJeXVcW6i+kWl13AL6AR3u3Ytaom0t+b033huDjTeDl3anfRE6SDz1vRVIFmsQa47CsRu8MHFhN+lCI8+MEaytuud/Fl8jwPkZYydr86jBL+3rFf4tiMEHdQxStmlLYFKEYtLGhYYhae7lKKTF1fcxMai+7ysa2Lpd6sXNusQvCZT4X4IYhFVF1VfAEk9zKJUaSNc43ltE1v7SoQRTfuBFj4zCLn/m//RvJ4syDwAUk6Z58bkLEAB73h34S8keSre2utsJ6byT3hq7sx7nXtA6XSjZVj/ruVEuLKnOPWNevD4w7siX5mX1R/jNF5zt8z7CZXKUBIiw1wkqj94f/rbROftEXWUXXLffzDMYddle4+8LIH6Ya+PbobAxjm682/qEhspjtIiyavCwMWsIrHkgiVNsDAoyPg5G/yTWjZsPysgbKNKK6muWjv6qvo5SVWC5jXYh8b3tX9ufW40iiuDh7S4Pg5b2v0S4Ih2/PlVUTWk34nqU=', 'smithi159.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJFWkONOX1m/IQoBLaXRPuRnuyN5L2gbWLlcTMGGh1dUsGISOV8Wfx4n0adCkI9Wh5FyrrqzkH2Jm+Fo70O9EBKuCMrtOTH+1CV1x4IAo+BEwC1VnKI5LTqzSADPmD67AZ8wEygDMFTlRQwR9Ok+/8V49Ky/CJ21ju9EwBoW/YD7pmKIIfnLC4falwhgkuAihXgCO4cUC+Z6/ndUt7s4St5V7Zz68gdlPWNWwldCC9Xsr0RurTSXt09tVxAcY2c59tPiLg7fuKKV7NpZZ/g5qxgP+t14jnb4YG4D5AmqpV+/oZ5BVHrhfkYn2C4OuaDZz+WOHLA8pPAvZAKFCpLZYpcHDF1POUI+9BbJs+VRLliI40emWs7fXCwqgLsPSjXbIhYeQgmtE1hGJ84MA6EA5SIXLiih/SYfENGDveLBONgxH5TCRdtnDla7ABueu34t3UDxQ+yotvwQqglxqnOl3XCkDeh68KVObvyy+Z7++NYJ0qoSDVmcWEq3Cz8XZkprc=', 'smithi165.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAy9i2EOsT8UOx7LgqTIHa8mfUt2zFMUe4jNDlRaCBnJHablrIgUN0sbyGOfRvtCSFidJ8HZPQL9qtfZW9fdmSw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'octopus', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, {'print': '**** done installing octopus'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cephfs': {'max_mds': 2}, 'flavor': 'default', 'fs': 'xfs', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client octopus']}}, {'print': '**** done ceph'}, {'ceph-fuse': None}, {'print': '**** done octopus client'}, {'workunit': {'clients': {'all': ['suites/fsstress.sh']}}}, {'print': '**** done fsstress'}, {'mds_pre_upgrade': None}, {'print': '**** done mds pre-upgrade sequence'}, {'install.upgrade': {'mon.a': {'branch': 'quincy'}, 'mon.b': {'branch': 'quincy'}}}, {'print': '**** done install.upgrade both hosts'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'ceph.stop': ['mds.*']}, {'ceph.restart': {'daemons': ['mds.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph osd dump -f json-pretty', 'ceph versions', 'ceph osd require-osd-release quincy', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}, {'workunit': {'clients': {'all': ['suites/fsstress.sh']}}}, {'print': '**** done fsstress'}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-23_17:24:26', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.20399'} 2022-11-27T17:01:41.629 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T17:01:41.644 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T17:01:41.651 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T17:01:41.669 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T17:01:41.704 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T17:01:41.708 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T17:01:41.736 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:41.736 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T17:01:41.787 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T17:01:41.788 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2022-11-27T17:01:41.845 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T17:01:41.845 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T17:01:41.845 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 378488 Links: 1 2022-11-27T17:01:41.845 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T17:01:41.846 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T17:01:41.846 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2022-11-27 17:01:34.163505151 +0000 2022-11-27T17:01:41.846 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2022-11-27 16:59:02.979885087 +0000 2022-11-27T17:01:41.846 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2022-11-27 16:59:02.979885087 +0000 2022-11-27T17:01:41.847 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2022-11-27T17:01:41.847 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T17:01:41.911 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2022-11-27T17:01:41.911 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2022-11-27T17:01:41.911 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000251413 s, 2.0 MB/s 2022-11-27T17:01:41.913 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T17:01:41.972 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2022-11-27T17:01:42.030 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T17:01:42.031 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T17:01:42.031 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 383019 Links: 1 2022-11-27T17:01:42.031 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T17:01:42.032 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T17:01:42.032 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2022-11-27 17:01:34.163505151 +0000 2022-11-27T17:01:42.032 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2022-11-27 16:59:03.543869725 +0000 2022-11-27T17:01:42.032 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2022-11-27 16:59:03.543869725 +0000 2022-11-27T17:01:42.033 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2022-11-27T17:01:42.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T17:01:42.096 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2022-11-27T17:01:42.097 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2022-11-27T17:01:42.097 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000193251 s, 2.6 MB/s 2022-11-27T17:01:42.100 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T17:01:42.159 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2022-11-27T17:01:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T17:01:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T17:01:42.217 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 379341 Links: 1 2022-11-27T17:01:42.217 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T17:01:42.218 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T17:01:42.218 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2022-11-27 17:01:34.163505151 +0000 2022-11-27T17:01:42.218 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2022-11-27 16:59:04.080855098 +0000 2022-11-27T17:01:42.219 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2022-11-27 16:59:04.080855098 +0000 2022-11-27T17:01:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2022-11-27T17:01:42.219 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T17:01:42.283 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2022-11-27T17:01:42.284 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2022-11-27T17:01:42.284 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000213069 s, 2.4 MB/s 2022-11-27T17:01:42.285 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T17:01:42.344 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2022-11-27T17:01:42.402 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T17:01:42.402 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T17:01:42.403 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 384024 Links: 1 2022-11-27T17:01:42.403 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T17:01:42.403 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T17:01:42.403 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2022-11-27 17:01:34.164505124 +0000 2022-11-27T17:01:42.404 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2022-11-27 16:59:04.607840743 +0000 2022-11-27T17:01:42.404 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2022-11-27 16:59:04.607840743 +0000 2022-11-27T17:01:42.404 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: - 2022-11-27T17:01:42.405 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T17:01:42.468 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2022-11-27T17:01:42.468 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2022-11-27T17:01:42.469 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000187006 s, 2.7 MB/s 2022-11-27T17:01:42.470 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T17:01:42.529 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T17:01:42.530 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2022-11-27T17:01:42.530 INFO:tasks.ceph:Generating config... 2022-11-27T17:01:42.532 INFO:tasks.ceph:[global] mon warn on pool no app = False 2022-11-27T17:01:42.532 INFO:tasks.ceph:[global] ms bind msgr2 = False 2022-11-27T17:01:42.532 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2022-11-27T17:01:42.533 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2022-11-27T17:01:42.533 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2022-11-27T17:01:42.533 INFO:tasks.ceph:[client] client mount timeout = 600 2022-11-27T17:01:42.534 INFO:tasks.ceph:[client] debug client = 20 2022-11-27T17:01:42.534 INFO:tasks.ceph:[client] debug ms = 1 2022-11-27T17:01:42.534 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-11-27T17:01:42.535 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-11-27T17:01:42.535 INFO:tasks.ceph:[mds] debug mds = 20 2022-11-27T17:01:42.535 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-11-27T17:01:42.536 INFO:tasks.ceph:[mds] debug ms = 1 2022-11-27T17:01:42.536 INFO:tasks.ceph:[mds] mds debug frag = True 2022-11-27T17:01:42.536 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-11-27T17:01:42.537 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-11-27T17:01:42.537 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-11-27T17:01:42.537 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-11-27T17:01:42.537 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-11-27T17:01:42.538 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-11-27T17:01:42.538 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-11-27T17:01:42.539 INFO:tasks.ceph:[mgr] debug ms = 1 2022-11-27T17:01:42.539 INFO:tasks.ceph:[mon] debug mon = 20 2022-11-27T17:01:42.539 INFO:tasks.ceph:[mon] debug ms = 1 2022-11-27T17:01:42.540 INFO:tasks.ceph:[mon] debug paxos = 20 2022-11-27T17:01:42.540 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-11-27T17:01:42.540 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2022-11-27T17:01:42.541 INFO:tasks.ceph:[osd] bdev async discard = True 2022-11-27T17:01:42.541 INFO:tasks.ceph:[osd] bdev enable discard = True 2022-11-27T17:01:42.541 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2022-11-27T17:01:42.541 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-11-27T17:01:42.542 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-11-27T17:01:42.542 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2022-11-27T17:01:42.542 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2022-11-27T17:01:42.543 INFO:tasks.ceph:[osd] debug ms = 1 2022-11-27T17:01:42.543 INFO:tasks.ceph:[osd] debug osd = 20 2022-11-27T17:01:42.543 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2022-11-27T17:01:42.544 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-11-27T17:01:42.544 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-11-27T17:01:42.544 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-11-27T17:01:42.545 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-11-27T17:01:42.545 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-11-27T17:01:42.545 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-11-27T17:01:42.546 INFO:tasks.ceph:Setting up mon.a... 2022-11-27T17:01:42.546 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-11-27T17:01:42.623 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.keyring 2022-11-27T17:01:42.625 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-11-27T17:01:42.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T17:01:42.734 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.154'), ('mon.b', '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]'), ('mon.c', '[v2:172.21.15.154:3302,v1:172.21.15.154:6791]')] 2022-11-27T17:01:42.735 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790],[v2:172.21.15.154:3302,v1:172.21.15.154:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds.c': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-11-27T17:01:42.736 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:42.737 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T17:01:42.792 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.154 --addv b '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]' --addv c '[v2:172.21.15.154:3302,v1:172.21.15.154:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2022-11-27T17:01:42.875 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-11-27T17:01:42.876 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: generated fsid 5adefcc8-72cd-433b-bd69-ec1eac0a6bf2 2022-11-27T17:01:42.876 INFO:teuthology.orchestra.run.smithi154.stdout:epoch 0 2022-11-27T17:01:42.876 INFO:teuthology.orchestra.run.smithi154.stdout:fsid 5adefcc8-72cd-433b-bd69-ec1eac0a6bf2 2022-11-27T17:01:42.877 INFO:teuthology.orchestra.run.smithi154.stdout:last_changed 2022-11-27T17:01:42.874205+0000 2022-11-27T17:01:42.877 INFO:teuthology.orchestra.run.smithi154.stdout:created 2022-11-27T17:01:42.874205+0000 2022-11-27T17:01:42.877 INFO:teuthology.orchestra.run.smithi154.stdout:min_mon_release 0 (unknown) 2022-11-27T17:01:42.877 INFO:teuthology.orchestra.run.smithi154.stdout:0: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.a 2022-11-27T17:01:42.878 INFO:teuthology.orchestra.run.smithi154.stdout:1: [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] mon.b 2022-11-27T17:01:42.878 INFO:teuthology.orchestra.run.smithi154.stdout:2: [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] mon.c 2022-11-27T17:01:42.878 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-11-27T17:01:42.880 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T17:01:42.937 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5adefcc8-72cd-433b-bd69-ec1eac0a6bf2... 2022-11-27T17:01:42.938 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T17:01:42.980 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T17:01:42.983 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T17:01:43.022 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2022-11-27T17:01:43.022 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2022-11-27T17:01:43.023 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T17:01:43.023 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2022-11-27T17:01:43.023 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.024 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2022-11-27T17:01:43.024 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.024 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2022-11-27T17:01:43.024 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.024 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2022-11-27T17:01:43.025 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2022-11-27T17:01:43.025 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.026 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2022-11-27T17:01:43.026 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2022-11-27T17:01:43.026 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.026 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T17:01:43.027 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2022-11-27T17:01:43.027 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.027 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T17:01:43.027 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T17:01:43.028 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.028 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2022-11-27T17:01:43.028 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.028 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2022-11-27T17:01:43.029 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2022-11-27T17:01:43.029 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2022-11-27T17:01:43.029 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2022-11-27T17:01:43.029 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2022-11-27T17:01:43.029 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2022-11-27T17:01:43.030 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2022-11-27T17:01:43.030 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T17:01:43.030 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T17:01:43.030 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2022-11-27T17:01:43.031 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.031 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T17:01:43.031 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.032 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2022-11-27T17:01:43.032 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.032 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2022-11-27T17:01:43.032 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.033 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2022-11-27T17:01:43.033 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2022-11-27T17:01:43.033 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2022-11-27T17:01:43.033 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790],[v2:172.21.15.154:3302,v1:172.21.15.154:6791]" 2022-11-27T17:01:43.034 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on pool no app = False 2022-11-27T17:01:43.034 INFO:teuthology.orchestra.run.smithi154.stdout: ms bind msgr2 = False 2022-11-27T17:01:43.034 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore warn on legacy statfs = False 2022-11-27T17:01:43.034 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore warn on no per pool omap = False 2022-11-27T17:01:43.034 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn min per osd = 0 2022-11-27T17:01:43.035 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 5adefcc8-72cd-433b-bd69-ec1eac0a6bf2 2022-11-27T17:01:43.035 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.035 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2022-11-27T17:01:43.036 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2022-11-27T17:01:43.036 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.036 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2022-11-27T17:01:43.036 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2022-11-27T17:01:43.036 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.037 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2022-11-27T17:01:43.037 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2022-11-27T17:01:43.037 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.037 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2022-11-27T17:01:43.038 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2022-11-27T17:01:43.038 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2022-11-27T17:01:43.038 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.038 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2022-11-27T17:01:43.039 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2022-11-27T17:01:43.039 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.039 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2022-11-27T17:01:43.040 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.040 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2022-11-27T17:01:43.040 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.040 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2022-11-27T17:01:43.041 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.041 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2022-11-27T17:01:43.041 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2022-11-27T17:01:43.041 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.042 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2022-11-27T17:01:43.042 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2022-11-27T17:01:43.042 INFO:teuthology.orchestra.run.smithi154.stdout: bdev async discard = True 2022-11-27T17:01:43.042 INFO:teuthology.orchestra.run.smithi154.stdout: bdev enable discard = True 2022-11-27T17:01:43.043 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore allocator = bitmap 2022-11-27T17:01:43.043 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2022-11-27T17:01:43.043 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2022-11-27T17:01:43.043 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 1/20 2022-11-27T17:01:43.044 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 1/20 2022-11-27T17:01:43.044 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2022-11-27T17:01:43.044 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2022-11-27T17:01:43.044 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 4/10 2022-11-27T17:01:43.044 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T17:01:43.045 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2022-11-27T17:01:43.045 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2022-11-27T17:01:43.045 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2022-11-27T17:01:43.045 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2022-11-27T17:01:43.046 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2022-11-27T17:01:43.046 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.046 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2022-11-27T17:01:43.046 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2022-11-27T17:01:43.046 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2022-11-27T17:01:43.047 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2022-11-27T17:01:43.047 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2022-11-27T17:01:43.047 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2022-11-27T17:01:43.047 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2022-11-27T17:01:43.048 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2022-11-27T17:01:43.048 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.048 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2022-11-27T17:01:43.049 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2022-11-27T17:01:43.049 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2022-11-27T17:01:43.049 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2022-11-27T17:01:43.049 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2022-11-27T17:01:43.050 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2022-11-27T17:01:43.050 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2022-11-27T17:01:43.050 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2022-11-27T17:01:43.050 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2022-11-27T17:01:43.051 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2022-11-27T17:01:43.051 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2022-11-27T17:01:43.052 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.052 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T17:01:43.053 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T17:01:43.053 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T17:01:43.053 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.054 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T17:01:43.054 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T17:01:43.055 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T17:01:43.055 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.056 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2022-11-27T17:01:43.056 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2022-11-27T17:01:43.057 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.057 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2022-11-27T17:01:43.058 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2022-11-27T17:01:43.058 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = False 2022-11-27T17:01:43.059 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:01:43.059 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2022-11-27T17:01:43.060 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2022-11-27T17:01:43.060 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2022-11-27T17:01:43.060 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2022-11-27T17:01:43.061 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T17:01:43.061 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T17:01:43.062 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2022-11-27T17:01:43.062 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2022-11-27T17:01:43.063 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2022-11-27T17:01:43.063 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2022-11-27T17:01:43.063 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2022-11-27T17:01:43.064 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2022-11-27T17:01:43.064 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2022-11-27T17:01:43.065 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2022-11-27T17:01:43.065 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2022-11-27T17:01:43.066 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2022-11-27T17:01:43.066 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2022-11-27T17:01:43.067 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2022-11-27T17:01:43.067 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2022-11-27T17:01:43.068 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2022-11-27T17:01:43.068 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2022-11-27T17:01:43.068 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2022-11-27T17:01:43.069 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = True 2022-11-27T17:01:43.069 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2022-11-27T17:01:43.070 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = True 2022-11-27T17:01:43.070 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2022-11-27T17:01:43.071 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2022-11-27T17:01:43.071 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2022-11-27T17:01:43.076 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2022-11-27T17:01:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2022-11-27T17:01:43.077 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T17:01:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2022-11-27T17:01:43.078 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2022-11-27T17:01:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.079 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2022-11-27T17:01:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.080 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2022-11-27T17:01:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2022-11-27T17:01:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.081 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2022-11-27T17:01:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2022-11-27T17:01:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T17:01:43.083 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2022-11-27T17:01:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T17:01:43.084 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T17:01:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2022-11-27T17:01:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.085 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2022-11-27T17:01:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2022-11-27T17:01:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2022-11-27T17:01:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2022-11-27T17:01:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2022-11-27T17:01:43.086 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2022-11-27T17:01:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2022-11-27T17:01:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T17:01:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T17:01:43.087 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2022-11-27T17:01:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T17:01:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.088 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2022-11-27T17:01:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2022-11-27T17:01:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.089 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2022-11-27T17:01:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2022-11-27T17:01:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2022-11-27T17:01:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = "172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790],[v2:172.21.15.154:3302,v1:172.21.15.154:6791]" 2022-11-27T17:01:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on pool no app = False 2022-11-27T17:01:43.090 INFO:teuthology.orchestra.run.smithi165.stdout: ms bind msgr2 = False 2022-11-27T17:01:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore warn on legacy statfs = False 2022-11-27T17:01:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore warn on no per pool omap = False 2022-11-27T17:01:43.091 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn min per osd = 0 2022-11-27T17:01:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 5adefcc8-72cd-433b-bd69-ec1eac0a6bf2 2022-11-27T17:01:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.092 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2022-11-27T17:01:43.092 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2022-11-27T17:01:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2022-11-27T17:01:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 600 2022-11-27T17:01:43.093 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2022-11-27T17:01:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2022-11-27T17:01:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.094 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2022-11-27T17:01:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2022-11-27T17:01:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2022-11-27T17:01:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.095 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2022-11-27T17:01:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2022-11-27T17:01:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2022-11-27T17:01:43.096 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2022-11-27T17:01:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2022-11-27T17:01:43.097 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2022-11-27T17:01:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2022-11-27T17:01:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2022-11-27T17:01:43.098 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2022-11-27T17:01:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: bdev async discard = True 2022-11-27T17:01:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: bdev enable discard = True 2022-11-27T17:01:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore allocator = bitmap 2022-11-27T17:01:43.099 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2022-11-27T17:01:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2022-11-27T17:01:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 1/20 2022-11-27T17:01:43.100 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 1/20 2022-11-27T17:01:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2022-11-27T17:01:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2022-11-27T17:01:43.101 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 4/10 2022-11-27T17:01:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T17:01:43.102 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2022-11-27T17:01:43.103 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2022-11-27T17:01:43.103 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2022-11-27T17:01:43.104 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2022-11-27T17:01:43.104 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2022-11-27T17:01:43.105 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.105 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2022-11-27T17:01:43.105 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2022-11-27T17:01:43.106 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2022-11-27T17:01:43.106 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2022-11-27T17:01:43.107 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2022-11-27T17:01:43.107 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2022-11-27T17:01:43.108 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2022-11-27T17:01:43.108 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2022-11-27T17:01:43.109 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.109 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2022-11-27T17:01:43.110 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2022-11-27T17:01:43.110 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2022-11-27T17:01:43.111 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2022-11-27T17:01:43.111 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2022-11-27T17:01:43.112 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2022-11-27T17:01:43.112 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2022-11-27T17:01:43.112 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2022-11-27T17:01:43.113 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2022-11-27T17:01:43.113 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2022-11-27T17:01:43.114 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2022-11-27T17:01:43.114 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.114 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T17:01:43.115 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T17:01:43.115 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T17:01:43.115 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.116 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T17:01:43.116 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T17:01:43.116 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T17:01:43.116 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.117 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2022-11-27T17:01:43.117 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2022-11-27T17:01:43.117 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.118 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2022-11-27T17:01:43.118 INFO:teuthology.orchestra.run.smithi165.stdout: mon op complaint time = 120 2022-11-27T17:01:43.118 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = False 2022-11-27T17:01:43.119 INFO:teuthology.orchestra.run.smithi165.stdout: 2022-11-27T17:01:43.119 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2022-11-27T17:01:43.119 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2022-11-27T17:01:43.119 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2022-11-27T17:01:43.120 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2022-11-27T17:01:43.120 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T17:01:43.120 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T17:01:43.120 INFO:teuthology.orchestra.run.smithi165.stdout: client mount timeout = 600 2022-11-27T17:01:43.121 INFO:teuthology.orchestra.run.smithi165.stdout: debug client = 20 2022-11-27T17:01:43.121 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2022-11-27T17:01:43.121 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2022-11-27T17:01:43.121 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2022-11-27T17:01:43.122 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2022-11-27T17:01:43.122 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.b] 2022-11-27T17:01:43.122 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.c] 2022-11-27T17:01:43.122 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.a] 2022-11-27T17:01:43.123 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.b] 2022-11-27T17:01:43.123 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.c] 2022-11-27T17:01:43.123 INFO:teuthology.orchestra.run.smithi165.stdout:[mds] 2022-11-27T17:01:43.123 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds = 20 2022-11-27T17:01:43.124 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds balancer = 20 2022-11-27T17:01:43.124 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2022-11-27T17:01:43.124 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug frag = True 2022-11-27T17:01:43.124 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug scatterstat = True 2022-11-27T17:01:43.125 INFO:teuthology.orchestra.run.smithi165.stdout: mds op complaint time = 180 2022-11-27T17:01:43.125 INFO:teuthology.orchestra.run.smithi165.stdout: mds verify scatter = True 2022-11-27T17:01:43.125 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2022-11-27T17:01:43.125 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2022-11-27T17:01:43.126 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2022-11-27T17:01:43.126 INFO:teuthology.orchestra.run.smithi159.stdout:[global] 2022-11-27T17:01:43.126 INFO:teuthology.orchestra.run.smithi159.stdout: chdir = "" 2022-11-27T17:01:43.127 INFO:teuthology.orchestra.run.smithi159.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T17:01:43.127 INFO:teuthology.orchestra.run.smithi159.stdout: auth supported = cephx 2022-11-27T17:01:43.127 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.128 INFO:teuthology.orchestra.run.smithi159.stdout: filestore xattr use omap = true 2022-11-27T17:01:43.128 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.128 INFO:teuthology.orchestra.run.smithi159.stdout: mon clock drift allowed = 1.000 2022-11-27T17:01:43.128 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.129 INFO:teuthology.orchestra.run.smithi159.stdout: osd crush chooseleaf type = 0 2022-11-27T17:01:43.129 INFO:teuthology.orchestra.run.smithi159.stdout: auth debug = true 2022-11-27T17:01:43.129 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.130 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on old message = true 2022-11-27T17:01:43.130 INFO:teuthology.orchestra.run.smithi159.stdout: ms die on bug = true 2022-11-27T17:01:43.130 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.130 INFO:teuthology.orchestra.run.smithi159.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T17:01:43.131 INFO:teuthology.orchestra.run.smithi159.stdout: mon pg warn max object skew = 0 2022-11-27T17:01:43.131 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.131 INFO:teuthology.orchestra.run.smithi159.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T17:01:43.131 INFO:teuthology.orchestra.run.smithi159.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T17:01:43.132 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.132 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default size = 2 2022-11-27T17:01:43.132 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.132 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow primary affinity = true 2022-11-27T17:01:43.132 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd allow pg remap = true 2022-11-27T17:01:43.133 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on legacy crush tunables = false 2022-11-27T17:01:43.133 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on crush straw calc version zero = false 2022-11-27T17:01:43.133 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on no sortbitwise = false 2022-11-27T17:01:43.133 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on osd down out interval zero = false 2022-11-27T17:01:43.134 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on too few osds = false 2022-11-27T17:01:43.134 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T17:01:43.134 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T17:01:43.135 INFO:teuthology.orchestra.run.smithi159.stdout: mon_allow_pool_size_one = true 2022-11-27T17:01:43.135 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.135 INFO:teuthology.orchestra.run.smithi159.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T17:01:43.136 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.136 INFO:teuthology.orchestra.run.smithi159.stdout: osd default data pool replay window = 5 2022-11-27T17:01:43.137 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.137 INFO:teuthology.orchestra.run.smithi159.stdout: mon allow pool delete = true 2022-11-27T17:01:43.138 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.138 INFO:teuthology.orchestra.run.smithi159.stdout: mon cluster log file level = debug 2022-11-27T17:01:43.139 INFO:teuthology.orchestra.run.smithi159.stdout: debug asserts on shutdown = true 2022-11-27T17:01:43.139 INFO:teuthology.orchestra.run.smithi159.stdout: mon health detail to clog = false 2022-11-27T17:01:43.139 INFO:teuthology.orchestra.run.smithi159.stdout: mon host = "172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790],[v2:172.21.15.154:3302,v1:172.21.15.154:6791]" 2022-11-27T17:01:43.140 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on pool no app = False 2022-11-27T17:01:43.140 INFO:teuthology.orchestra.run.smithi159.stdout: ms bind msgr2 = False 2022-11-27T17:01:43.141 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore warn on legacy statfs = False 2022-11-27T17:01:43.141 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore warn on no per pool omap = False 2022-11-27T17:01:43.142 INFO:teuthology.orchestra.run.smithi159.stdout: mon pg warn min per osd = 0 2022-11-27T17:01:43.142 INFO:teuthology.orchestra.run.smithi159.stdout: fsid = 5adefcc8-72cd-433b-bd69-ec1eac0a6bf2 2022-11-27T17:01:43.142 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.143 INFO:teuthology.orchestra.run.smithi159.stdout:[osd] 2022-11-27T17:01:43.143 INFO:teuthology.orchestra.run.smithi159.stdout: osd journal size = 100 2022-11-27T17:01:43.143 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.144 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub load threshold = 5.0 2022-11-27T17:01:43.144 INFO:teuthology.orchestra.run.smithi159.stdout: osd scrub max interval = 600 2022-11-27T17:01:43.145 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.145 INFO:teuthology.orchestra.run.smithi159.stdout: osd recover clone overlap = true 2022-11-27T17:01:43.145 INFO:teuthology.orchestra.run.smithi159.stdout: osd recovery max chunk = 1048576 2022-11-27T17:01:43.146 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.146 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug shutdown = true 2022-11-27T17:01:43.146 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug op order = true 2022-11-27T17:01:43.147 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug verify stray on activate = true 2022-11-27T17:01:43.147 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.147 INFO:teuthology.orchestra.run.smithi159.stdout: osd open classes on start = true 2022-11-27T17:01:43.147 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug pg log writeout = true 2022-11-27T17:01:43.147 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.148 INFO:teuthology.orchestra.run.smithi159.stdout: osd deep scrub update digest min age = 30 2022-11-27T17:01:43.148 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.148 INFO:teuthology.orchestra.run.smithi159.stdout: osd map max advance = 10 2022-11-27T17:01:43.148 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.149 INFO:teuthology.orchestra.run.smithi159.stdout: journal zero on create = true 2022-11-27T17:01:43.149 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.149 INFO:teuthology.orchestra.run.smithi159.stdout: filestore ondisk finisher threads = 3 2022-11-27T17:01:43.149 INFO:teuthology.orchestra.run.smithi159.stdout: filestore apply finisher threads = 3 2022-11-27T17:01:43.149 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.150 INFO:teuthology.orchestra.run.smithi159.stdout: bdev debug aio = true 2022-11-27T17:01:43.150 INFO:teuthology.orchestra.run.smithi159.stdout: osd debug misdirected ops = true 2022-11-27T17:01:43.150 INFO:teuthology.orchestra.run.smithi159.stdout: bdev async discard = True 2022-11-27T17:01:43.151 INFO:teuthology.orchestra.run.smithi159.stdout: bdev enable discard = True 2022-11-27T17:01:43.151 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore allocator = bitmap 2022-11-27T17:01:43.151 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore block size = 96636764160 2022-11-27T17:01:43.151 INFO:teuthology.orchestra.run.smithi159.stdout: bluestore fsck on mount = True 2022-11-27T17:01:43.152 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluefs = 1/20 2022-11-27T17:01:43.152 INFO:teuthology.orchestra.run.smithi159.stdout: debug bluestore = 1/20 2022-11-27T17:01:43.152 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2022-11-27T17:01:43.153 INFO:teuthology.orchestra.run.smithi159.stdout: debug osd = 20 2022-11-27T17:01:43.153 INFO:teuthology.orchestra.run.smithi159.stdout: debug rocksdb = 4/10 2022-11-27T17:01:43.153 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T17:01:43.153 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd full ratio = 0.9 2022-11-27T17:01:43.154 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd nearfull ratio = 0.8 2022-11-27T17:01:43.154 INFO:teuthology.orchestra.run.smithi159.stdout: osd failsafe full ratio = 0.95 2022-11-27T17:01:43.154 INFO:teuthology.orchestra.run.smithi159.stdout: osd objectstore = bluestore 2022-11-27T17:01:43.154 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2022-11-27T17:01:43.155 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.155 INFO:teuthology.orchestra.run.smithi159.stdout:[mgr] 2022-11-27T17:01:43.155 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2022-11-27T17:01:43.155 INFO:teuthology.orchestra.run.smithi159.stdout: debug mgr = 20 2022-11-27T17:01:43.156 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2022-11-27T17:01:43.156 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2022-11-27T17:01:43.156 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2022-11-27T17:01:43.156 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2022-11-27T17:01:43.156 INFO:teuthology.orchestra.run.smithi159.stdout: mgr/telemetry/nag = false 2022-11-27T17:01:43.157 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.157 INFO:teuthology.orchestra.run.smithi159.stdout:[mon] 2022-11-27T17:01:43.157 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2022-11-27T17:01:43.157 INFO:teuthology.orchestra.run.smithi159.stdout: debug mon = 20 2022-11-27T17:01:43.158 INFO:teuthology.orchestra.run.smithi159.stdout: debug paxos = 20 2022-11-27T17:01:43.158 INFO:teuthology.orchestra.run.smithi159.stdout: debug auth = 20 2022-11-27T17:01:43.158 INFO:teuthology.orchestra.run.smithi159.stdout: mon data avail warn = 5 2022-11-27T17:01:43.159 INFO:teuthology.orchestra.run.smithi159.stdout: mon mgr mkfs grace = 240 2022-11-27T17:01:43.159 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min pgs per osd = 4 2022-11-27T17:01:43.159 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd reporter subtree level = osd 2022-11-27T17:01:43.159 INFO:teuthology.orchestra.run.smithi159.stdout: mon osd prime pg temp = true 2022-11-27T17:01:43.160 INFO:teuthology.orchestra.run.smithi159.stdout: mon reweight min bytes per osd = 10 2022-11-27T17:01:43.160 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.160 INFO:teuthology.orchestra.run.smithi159.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T17:01:43.160 INFO:teuthology.orchestra.run.smithi159.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T17:01:43.161 INFO:teuthology.orchestra.run.smithi159.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T17:01:43.161 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.161 INFO:teuthology.orchestra.run.smithi159.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T17:01:43.162 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T17:01:43.162 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T17:01:43.162 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.162 INFO:teuthology.orchestra.run.smithi159.stdout: # 1m isn't quite enough 2022-11-27T17:01:43.163 INFO:teuthology.orchestra.run.smithi159.stdout: mon_down_mkfs_grace = 2m 2022-11-27T17:01:43.163 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.163 INFO:teuthology.orchestra.run.smithi159.stdout: mon_warn_on_filestore_osds = false 2022-11-27T17:01:43.163 INFO:teuthology.orchestra.run.smithi159.stdout: mon op complaint time = 120 2022-11-27T17:01:43.163 INFO:teuthology.orchestra.run.smithi159.stdout: mon warn on osd down out interval zero = False 2022-11-27T17:01:43.164 INFO:teuthology.orchestra.run.smithi159.stdout: 2022-11-27T17:01:43.164 INFO:teuthology.orchestra.run.smithi159.stdout:[client] 2022-11-27T17:01:43.164 INFO:teuthology.orchestra.run.smithi159.stdout: rgw cache enabled = true 2022-11-27T17:01:43.164 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable ops log = true 2022-11-27T17:01:43.165 INFO:teuthology.orchestra.run.smithi159.stdout: rgw enable usage log = true 2022-11-27T17:01:43.165 INFO:teuthology.orchestra.run.smithi159.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T17:01:43.165 INFO:teuthology.orchestra.run.smithi159.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T17:01:43.165 INFO:teuthology.orchestra.run.smithi159.stdout: client mount timeout = 600 2022-11-27T17:01:43.166 INFO:teuthology.orchestra.run.smithi159.stdout: debug client = 20 2022-11-27T17:01:43.166 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2022-11-27T17:01:43.166 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2022-11-27T17:01:43.166 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2022-11-27T17:01:43.167 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.a] 2022-11-27T17:01:43.167 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.b] 2022-11-27T17:01:43.167 INFO:teuthology.orchestra.run.smithi159.stdout:[mon.c] 2022-11-27T17:01:43.174 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.a] 2022-11-27T17:01:43.174 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.b] 2022-11-27T17:01:43.174 INFO:teuthology.orchestra.run.smithi159.stdout:[mds.c] 2022-11-27T17:01:43.175 INFO:teuthology.orchestra.run.smithi159.stdout:[mds] 2022-11-27T17:01:43.175 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds = 20 2022-11-27T17:01:43.176 INFO:teuthology.orchestra.run.smithi159.stdout: debug mds balancer = 20 2022-11-27T17:01:43.176 INFO:teuthology.orchestra.run.smithi159.stdout: debug ms = 1 2022-11-27T17:01:43.176 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug frag = True 2022-11-27T17:01:43.177 INFO:teuthology.orchestra.run.smithi159.stdout: mds debug scatterstat = True 2022-11-27T17:01:43.177 INFO:teuthology.orchestra.run.smithi159.stdout: mds op complaint time = 180 2022-11-27T17:01:43.177 INFO:teuthology.orchestra.run.smithi159.stdout: mds verify scatter = True 2022-11-27T17:01:43.178 INFO:teuthology.orchestra.run.smithi159.stdout: osd op complaint time = 180 2022-11-27T17:01:43.178 INFO:teuthology.orchestra.run.smithi159.stdout: rados mon op timeout = 900 2022-11-27T17:01:43.179 INFO:teuthology.orchestra.run.smithi159.stdout: rados osd op timeout = 900 2022-11-27T17:01:43.179 INFO:tasks.ceph:Creating admin key on mon.a... 2022-11-27T17:01:43.180 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-11-27T17:01:43.232 INFO:tasks.ceph:Copying monmap to all nodes... 2022-11-27T17:01:43.232 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:43.233 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-11-27T17:01:43.252 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:43.253 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-11-27T17:01:43.310 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2022-11-27T17:01:43.311 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:43.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T17:01:43.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T17:01:43.387 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:43.387 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T17:01:43.446 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi159.front.sepia.ceph.com 2022-11-27T17:01:43.446 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2022-11-27T17:01:43.447 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T17:01:43.447 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T17:01:43.485 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2022-11-27T17:01:43.485 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T17:01:43.542 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi165.front.sepia.ceph.com 2022-11-27T17:01:43.543 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2022-11-27T17:01:43.543 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T17:01:43.544 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T17:01:43.581 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2022-11-27T17:01:43.582 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T17:01:43.639 INFO:tasks.ceph:Setting up mon nodes... 2022-11-27T17:01:43.639 INFO:tasks.ceph:Setting up mgr nodes... 2022-11-27T17:01:43.640 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T17:01:43.689 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T17:01:43.692 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T17:01:43.740 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T17:01:43.742 INFO:tasks.ceph:Setting up mds nodes... 2022-11-27T17:01:43.743 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T17:01:43.792 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T17:01:43.794 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-11-27T17:01:43.822 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T17:01:43.911 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T17:01:43.950 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-11-27T17:01:43.977 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T17:01:44.066 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T17:01:44.068 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-11-27T17:01:44.095 INFO:tasks.ceph_client:Setting up client nodes... 2022-11-27T17:01:44.096 DEBUG:teuthology.orchestra.run.smithi159:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-11-27T17:01:44.145 INFO:teuthology.orchestra.run.smithi159.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-11-27T17:01:44.157 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-11-27T17:01:44.208 INFO:teuthology.orchestra.run.smithi165.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-11-27T17:01:44.221 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-11-27T17:01:44.221 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2022-11-27T17:01:44.222 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-11-27T17:01:44.249 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T17:01:44.249 INFO:tasks.ceph:role: osd.0 2022-11-27T17:01:44.250 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2022-11-27T17:01:44.250 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T17:01:44.323 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T17:01:44.324 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T17:01:44.324 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T17:01:44.324 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T17:01:44.325 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T17:01:44.325 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2022-11-27T17:01:44.325 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T17:01:44.325 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T17:01:44.326 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T17:01:44.326 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T17:01:44.768 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2022-11-27T17:01:44.770 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2022-11-27T17:01:44.770 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-11-27T17:01:44.941 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-11-27T17:01:44.975 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-11-27T17:01:45.044 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T17:01:45.044 INFO:tasks.ceph:role: osd.1 2022-11-27T17:01:45.045 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2022-11-27T17:01:45.045 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T17:01:45.119 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T17:01:45.119 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T17:01:45.119 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T17:01:45.120 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T17:01:45.120 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T17:01:45.120 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2022-11-27T17:01:45.120 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T17:01:45.121 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T17:01:45.121 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T17:01:45.121 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T17:01:45.541 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2022-11-27T17:01:45.543 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2022-11-27T17:01:45.543 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-11-27T17:01:45.648 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-11-27T17:01:45.680 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-11-27T17:01:45.747 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T17:01:45.747 INFO:tasks.ceph:role: osd.2 2022-11-27T17:01:45.748 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2022-11-27T17:01:45.748 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T17:01:45.821 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T17:01:45.822 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T17:01:45.823 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T17:01:45.823 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T17:01:45.824 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T17:01:45.824 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2022-11-27T17:01:45.824 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T17:01:45.825 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T17:01:45.825 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T17:01:45.826 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T17:01:46.264 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2022-11-27T17:01:46.265 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2022-11-27T17:01:46.266 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-11-27T17:01:46.426 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-11-27T17:01:46.458 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-11-27T17:01:46.525 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T17:01:46.525 INFO:tasks.ceph:role: osd.3 2022-11-27T17:01:46.525 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2022-11-27T17:01:46.526 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T17:01:46.600 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T17:01:46.601 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T17:01:46.601 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T17:01:46.602 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T17:01:46.602 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T17:01:46.603 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2022-11-27T17:01:46.603 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T17:01:46.604 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T17:01:46.604 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T17:01:46.604 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T17:01:47.036 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2022-11-27T17:01:47.062 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2022-11-27T17:01:47.062 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-11-27T17:01:47.263 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-11-27T17:01:47.295 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T17:01:47.379 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:47.376+0000 7fe1f14c1f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-11-27T17:01:47.380 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:47.376+0000 7fe1f14c1f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-11-27T17:01:47.380 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:47.376+0000 7fe1f14c1f00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T17:01:47.381 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:47.376+0000 7fe1f14c1f00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T17:01:47.381 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:47.376+0000 7fe1f14c1f00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T17:01:47.382 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:47.377+0000 7fe1f14c1f00 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-11-27T17:01:47.391 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:47.389+0000 7fe1f14c1f00 -1 freelist read_size_meta_from_db missing size meta in DB 2022-11-27T17:01:49.708 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-11-27T17:01:49.779 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T17:01:49.862 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:49.860+0000 7f2f8b5cef00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-11-27T17:01:49.863 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:49.860+0000 7f2f8b5cef00 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-11-27T17:01:49.863 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:49.860+0000 7f2f8b5cef00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T17:01:49.864 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:49.860+0000 7f2f8b5cef00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T17:01:49.864 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:49.860+0000 7f2f8b5cef00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T17:01:49.864 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:49.861+0000 7f2f8b5cef00 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-11-27T17:01:49.872 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:49.871+0000 7f2f8b5cef00 -1 freelist read_size_meta_from_db missing size meta in DB 2022-11-27T17:01:52.191 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-11-27T17:01:52.261 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T17:01:52.347 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:52.344+0000 7f0bcb61cf00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-11-27T17:01:52.347 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:52.344+0000 7f0bcb61cf00 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-11-27T17:01:52.348 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:52.344+0000 7f0bcb61cf00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T17:01:52.348 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:52.344+0000 7f0bcb61cf00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T17:01:52.348 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:52.344+0000 7f0bcb61cf00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T17:01:52.348 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:52.344+0000 7f0bcb61cf00 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-11-27T17:01:52.354 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:52.351+0000 7f0bcb61cf00 -1 freelist read_size_meta_from_db missing size meta in DB 2022-11-27T17:01:54.676 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-11-27T17:01:54.706 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T17:01:54.791 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:54.787+0000 7fd7f2652f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-11-27T17:01:54.791 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:54.788+0000 7fd7f2652f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-11-27T17:01:54.791 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:54.788+0000 7fd7f2652f00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T17:01:54.792 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:54.788+0000 7fd7f2652f00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T17:01:54.792 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:54.788+0000 7fd7f2652f00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T17:01:54.792 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:54.788+0000 7fd7f2652f00 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-11-27T17:01:54.797 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:54.794+0000 7fd7f2652f00 -1 freelist read_size_meta_from_db missing size meta in DB 2022-11-27T17:01:57.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-11-27T17:01:57.146 INFO:tasks.ceph:Reading keys from all nodes... 2022-11-27T17:01:57.147 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:57.147 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-11-27T17:01:57.214 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:57.215 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-11-27T17:01:57.292 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:57.292 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-11-27T17:01:57.358 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:57.359 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-11-27T17:01:57.425 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:57.426 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-11-27T17:01:57.493 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:57.493 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-11-27T17:01:57.560 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:57.561 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-11-27T17:01:57.627 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:57.628 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-11-27T17:01:57.694 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:57.695 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-11-27T17:01:57.761 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2022-11-27T17:01:57.762 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-11-27T17:01:57.783 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2022-11-27T17:01:57.784 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-11-27T17:01:57.804 INFO:tasks.ceph:Adding keys to all mons... 2022-11-27T17:01:57.805 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T17:01:57.831 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2022-11-27T17:01:57.832 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQD3l4NjnVcKKRAAunyIfm9ame6hNnP+shpXZg== 2022-11-27T17:01:57.832 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2022-11-27T17:01:57.832 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQD3l4NjidUQLBAAmL8wStTa1nj4c/KLg9ca8Q== 2022-11-27T17:01:57.832 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2022-11-27T17:01:57.832 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQD3l4NjiFYnLxAAR8b5RMJH2YkdUbvM9OSYnQ== 2022-11-27T17:01:57.833 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2022-11-27T17:01:57.833 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQD3l4NjuwhDNhAADBvceYLFRan8RAPtvzPn4g== 2022-11-27T17:01:57.833 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2022-11-27T17:01:57.833 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQD4l4NjHiflAxAAg+ulqIXdVLxo2H57R7jNdg== 2022-11-27T17:01:57.834 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2022-11-27T17:01:57.834 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQD7l4NjIdV2FhAAW4k/6zfZ78U/36D10SdfSg== 2022-11-27T17:01:57.834 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2022-11-27T17:01:57.834 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQD9l4NjlElOMxAA0oanypDdFowIsLl5v4WpBA== 2022-11-27T17:01:57.835 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2022-11-27T17:01:57.835 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAAmINju6KUFBAA6J7vQd6FRZ+aF1Vclwi2Tg== 2022-11-27T17:01:57.835 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2022-11-27T17:01:57.835 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQACmINjn94GLxAA1B2zOL7CKvmjBd7qM1Z1Kw== 2022-11-27T17:01:57.835 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2022-11-27T17:01:57.836 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQD4l4NjsIGTCBAA2pLdSMp+fZNjzh2e1clwmg== 2022-11-27T17:01:57.836 INFO:teuthology.orchestra.run.smithi154.stdout:[client.1] 2022-11-27T17:01:57.836 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQD4l4NjL6JgDBAA4W1e72p8z7DPmINRcfNI3A== 2022-11-27T17:01:57.837 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T17:01:57.926 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T17:01:57.975 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T17:01:58.026 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T17:01:58.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T17:01:58.129 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T17:01:58.178 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T17:01:58.226 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T17:01:58.274 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T17:01:58.323 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T17:01:58.374 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T17:01:58.425 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-11-27T17:01:58.426 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-11-27T17:01:58.454 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T17:01:58.548 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-11-27T17:01:58.575 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-11-27T17:01:58.643 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T17:01:58.736 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-11-27T17:01:58.765 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-11-27T17:01:58.832 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T17:01:58.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-11-27T17:01:58.957 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T17:01:59.014 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-11-27T17:01:59.016 INFO:tasks.ceph.mon.a:Restarting daemon 2022-11-27T17:01:59.016 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-11-27T17:01:59.058 INFO:tasks.ceph.mon.a:Started 2022-11-27T17:01:59.059 INFO:tasks.ceph.mon.b:Restarting daemon 2022-11-27T17:01:59.060 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-11-27T17:01:59.062 INFO:tasks.ceph.mon.b:Started 2022-11-27T17:01:59.062 INFO:tasks.ceph.mon.c:Restarting daemon 2022-11-27T17:01:59.063 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-11-27T17:01:59.065 INFO:tasks.ceph.mon.c:Started 2022-11-27T17:01:59.067 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-11-27T17:01:59.069 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-11-27T17:01:59.070 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-11-27T17:01:59.073 INFO:tasks.ceph.mgr.x:Started 2022-11-27T17:01:59.073 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-11-27T17:01:59.074 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-11-27T17:01:59.077 INFO:tasks.ceph.mgr.y:Started 2022-11-27T17:01:59.077 INFO:tasks.ceph:Setting crush tunables to default 2022-11-27T17:01:59.078 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd crush tunables default 2022-11-27T17:01:59.217 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.213+0000 7f942a6c2700 1 Processor -- start 2022-11-27T17:01:59.218 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.213+0000 7f942a6c2700 1 -- start start 2022-11-27T17:01:59.218 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.213+0000 7f942a6c2700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9424099400 0x7f9424139c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.218 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.213+0000 7f942a6c2700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f942413a150 0x7f9424140e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.219 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.213+0000 7f942a6c2700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9424099bd0 con 0x7f9424139880 2022-11-27T17:01:59.219 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.213+0000 7f942a6c2700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9424097f20 con 0x7f942413a150 2022-11-27T17:01:59.219 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.213+0000 7f942a6c2700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9424097c80 con 0x7f9424099400 2022-11-27T17:01:59.219 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f94237fe700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9424099400 0x7f9424139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.220 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f94237fe700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9424099400 0x7f9424139c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:54710/0 (socket says 172.21.15.154:54710) 2022-11-27T17:01:59.220 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f94237fe700 1 -- 172.21.15.154:0/8533532 learned_addr learned my addr 172.21.15.154:0/8533532 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:01:59.220 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f9428c5f700 1 --2- 172.21.15.154:0/8533532 >> v2:172.21.15.154:3300/0 conn(0x7f942413a150 0x7f9424140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.220 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f94237fe700 1 -- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9424099400 msgr2=0x7f9424139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T17:01:59.221 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f9428c5f700 1 -- 172.21.15.154:0/8533532 >> v2:172.21.15.154:3300/0 conn(0x7f942413a150 msgr2=0x7f9424140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-11-27T17:01:59.221 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f94237fe700 1 -- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9424099400 msgr2=0x7f9424139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T17:01:59.221 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f9428c5f700 1 -- 172.21.15.154:0/8533532 >> v2:172.21.15.154:3300/0 conn(0x7f942413a150 msgr2=0x7f9424140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T17:01:59.222 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f94237fe700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9424099400 0x7f9424139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-27T17:01:59.222 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f9428c5f700 1 --2- 172.21.15.154:0/8533532 >> v2:172.21.15.154:3300/0 conn(0x7f942413a150 0x7f9424140e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-27T17:01:59.222 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f94237fe700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9424099400 0x7f9424139c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-11-27T17:01:59.222 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.214+0000 7f9428c5f700 1 --2- 172.21.15.154:0/8533532 >> v2:172.21.15.154:3300/0 conn(0x7f942413a150 0x7f9424140e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-11-27T17:01:59.386 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.382+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186786584 0 0) 0x7f9424099bd0 con 0x7f9424139880 2022-11-27T17:01:59.387 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.382+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9410003260 con 0x7f9424139880 2022-11-27T17:01:59.387 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1261909638 0 0) 0x7f9410003260 con 0x7f9424139880 2022-11-27T17:01:59.388 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9424099400 msgr2=0x7f9424139c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T17:01:59.388 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9424099400 0x7f9424139c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.388 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 >> v2:172.21.15.154:3300/0 conn(0x7f942413a150 msgr2=0x7f9424140e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T17:01:59.388 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 --2- 172.21.15.154:0/8533532 >> v2:172.21.15.154:3300/0 conn(0x7f942413a150 0x7f9424140e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.388 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942405cb00 con 0x7f9424139880 2022-11-27T17:01:59.389 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 1207237276 0 0) 0x7f9414001660 con 0x7f9424139880 2022-11-27T17:01:59.389 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 >> v1:172.21.15.154:6789/0 conn(0x7f9424139880 legacy=0x7f94240974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:01:59.389 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9410003780 0x7f9410003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.389 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9410004500 0x7f9410004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.390 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9410004e70 0x7f941000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.390 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9424097c80 con 0x7f9410004e70 2022-11-27T17:01:59.390 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941000bcf0 con 0x7f9410003780 2022-11-27T17:01:59.390 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f941000beb0 con 0x7f9410004500 2022-11-27T17:01:59.391 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9423fff700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9410003780 0x7f9410003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.391 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9423fff700 1 -- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9410004500 msgr2=0x7f9410004980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T17:01:59.391 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9423fff700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f9410004500 0x7f9410004980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-11-27T17:01:59.391 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9423fff700 1 -- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9410004e70 msgr2=0x7f941000b750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2022-11-27T17:01:59.392 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9423fff700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9410004e70 0x7f941000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-11-27T17:01:59.392 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9423fff700 1 -- 172.21.15.154:0/8533532 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410003260 con 0x7f9410003780 2022-11-27T17:01:59.392 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.383+0000 7f9423fff700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9410003780 0x7f9410003ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f94100054d0 tx=0x7f9414012870).ready entity=mon.1 client_cookie=7ad2569d23216842 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:01:59.396 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.385+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9414017070 con 0x7f9410003780 2022-11-27T17:01:59.396 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.385+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9414001be0 con 0x7f9410003780 2022-11-27T17:01:59.396 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.385+0000 7f9422ffd700 1 -- 172.21.15.154:0/8533532 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94140120d0 con 0x7f9410003780 2022-11-27T17:01:59.396 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.385+0000 7f942a6c2700 1 -- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9410003780 msgr2=0x7f9410003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:01:59.397 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.385+0000 7f942a6c2700 1 --2- 172.21.15.154:0/8533532 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9410003780 0x7f9410003ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f94100054d0 tx=0x7f9414012870).stop 2022-11-27T17:01:59.397 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.385+0000 7f9428c5f700 1 -- 172.21.15.154:0/8533532 reap_dead start 2022-11-27T17:01:59.397 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.386+0000 7f942a6c2700 1 -- 172.21.15.154:0/8533532 shutdown_connections 2022-11-27T17:01:59.397 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.386+0000 7f942a6c2700 1 -- 172.21.15.154:0/8533532 >> 172.21.15.154:0/8533532 conn(0x7f942405c4f0 msgr2=0x7f94240a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:01:59.398 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.386+0000 7f942a6c2700 1 -- 172.21.15.154:0/8533532 shutdown_connections 2022-11-27T17:01:59.398 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.386+0000 7f942a6c2700 1 -- 172.21.15.154:0/8533532 wait complete. 2022-11-27T17:01:59.398 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.387+0000 7f942a6c2700 1 Processor -- start 2022-11-27T17:01:59.398 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.387+0000 7f942a6c2700 1 -- start start 2022-11-27T17:01:59.398 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f942a6c2700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9410003780 0x7f9424063200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.399 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f942a6c2700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f94240636f0 0x7f9424141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.399 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f942a6c2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9424141b20 0x7f94240960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.399 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f942a6c2700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94241412b0 con 0x7f9410003780 2022-11-27T17:01:59.399 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f942a6c2700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9424065000 con 0x7f9424141b20 2022-11-27T17:01:59.400 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f942a6c2700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9424063e90 con 0x7f94240636f0 2022-11-27T17:01:59.400 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f94237fe700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f94240636f0 0x7f9424141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.400 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f94237fe700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f94240636f0 0x7f9424141630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:54778/0 (socket says 172.21.15.154:54778) 2022-11-27T17:01:59.402 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f94237fe700 1 -- 172.21.15.154:0/3045765363 learned_addr learned my addr 172.21.15.154:0/3045765363 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:01:59.402 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f9423fff700 1 --2- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9410003780 0x7f9424063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.403 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f94237fe700 1 -- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9424141b20 msgr2=0x7f94240960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:01:59.404 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f94237fe700 1 --2- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9424141b20 0x7f94240960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-11-27T17:01:59.405 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f94237fe700 1 -- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9410003780 msgr2=0x7f9424063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:01:59.406 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f94237fe700 1 --2- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9410003780 0x7f9424063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.407 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f94237fe700 1 -- 172.21.15.154:0/3045765363 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424064360 con 0x7f94240636f0 2022-11-27T17:01:59.407 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f9423fff700 1 --2- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9410003780 0x7f9424063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:01:59.407 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.388+0000 7f94237fe700 1 --2- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f94240636f0 0x7f9424141630 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f9424065740 tx=0x7f9418008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:01:59.407 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.389+0000 7f94217fa700 1 -- 172.21.15.154:0/3045765363 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9418013670 con 0x7f94240636f0 2022-11-27T17:01:59.408 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.389+0000 7f94217fa700 1 -- 172.21.15.154:0/3045765363 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94180137d0 con 0x7f94240636f0 2022-11-27T17:01:59.408 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.389+0000 7f942a6c2700 1 -- 172.21.15.154:0/3045765363 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9424061f40 con 0x7f94240636f0 2022-11-27T17:01:59.408 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.389+0000 7f94217fa700 1 -- 172.21.15.154:0/3045765363 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9418013b30 con 0x7f94240636f0 2022-11-27T17:01:59.408 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.389+0000 7f942a6c2700 1 -- 172.21.15.154:0/3045765363 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9424062680 con 0x7f94240636f0 2022-11-27T17:01:59.409 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.389+0000 7f94217fa700 1 -- 172.21.15.154:0/3045765363 <== mon.2 v2:172.21.15.154:3302/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f941801c5c0 con 0x7f94240636f0 2022-11-27T17:01:59.409 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.389+0000 7f94217fa700 1 -- 172.21.15.154:0/3045765363 <== mon.2 v2:172.21.15.154:3302/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9418000e40 con 0x7f94240636f0 2022-11-27T17:01:59.409 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.390+0000 7f942a6c2700 1 -- 172.21.15.154:0/3045765363 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc003400 con 0x7f94240636f0 2022-11-27T17:01:59.409 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.393+0000 7f94217fa700 1 -- 172.21.15.154:0/3045765363 <== mon.2 v2:172.21.15.154:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f9418012730 con 0x7f94240636f0 2022-11-27T17:01:59.476 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.473+0000 7f942a6c2700 1 -- 172.21.15.154:0/3045765363 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f93fc0036b0 con 0x7f94240636f0 2022-11-27T17:01:59.480 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.478+0000 7f94217fa700 1 -- 172.21.15.154:0/3045765363 <== mon.2 v2:172.21.15.154:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f94180128d0 con 0x7f94240636f0 2022-11-27T17:01:59.482 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.479+0000 7f942a6c2700 1 -- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f94240636f0 msgr2=0x7f9424141630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:01:59.482 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.479+0000 7f942a6c2700 1 --2- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f94240636f0 0x7f9424141630 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f9424065740 tx=0x7f9418008040).stop 2022-11-27T17:01:59.483 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.479+0000 7f942a6c2700 1 -- 172.21.15.154:0/3045765363 shutdown_connections 2022-11-27T17:01:59.483 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.479+0000 7f942a6c2700 1 --2- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f9424141b20 0x7f94240960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-11-27T17:01:59.483 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.479+0000 7f942a6c2700 1 --2- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9410003780 0x7f9424063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.483 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.479+0000 7f942a6c2700 1 --2- 172.21.15.154:0/3045765363 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f94240636f0 0x7f9424141630 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.484 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.479+0000 7f942a6c2700 1 -- 172.21.15.154:0/3045765363 >> 172.21.15.154:0/3045765363 conn(0x7f942405c4f0 msgr2=0x7f9424136b20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:01:59.484 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.479+0000 7f942a6c2700 1 -- 172.21.15.154:0/3045765363 shutdown_connections 2022-11-27T17:01:59.484 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.479+0000 7f942a6c2700 1 -- 172.21.15.154:0/3045765363 wait complete. 2022-11-27T17:01:59.484 INFO:teuthology.orchestra.run.smithi154.stderr:adjusted tunables profile to default 2022-11-27T17:01:59.492 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-11-27T17:01:59.493 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:59.493 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-11-27T17:01:59.521 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:59.522 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-11-27T17:01:59.591 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:59.592 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-11-27T17:01:59.661 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2022-11-27T17:01:59.661 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-11-27T17:01:59.728 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 3061801a-e670-4a13-9a7d-6c526f88ef28 0 2022-11-27T17:01:59.877 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.868+0000 7f1a4eb27700 1 Processor -- start 2022-11-27T17:01:59.878 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.868+0000 7f1a4eb27700 1 -- start start 2022-11-27T17:01:59.878 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4eb27700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a48139880 0x7f1a48099400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.878 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4eb27700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a480974a0 0x7f1a48139c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.879 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4eb27700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1a4813a150 0x7f1a481330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.879 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4eb27700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a48097f20 con 0x7f1a4813a150 2022-11-27T17:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4eb27700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48099bd0 con 0x7f1a480974a0 2022-11-27T17:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4eb27700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48097c80 con 0x7f1a48139880 2022-11-27T17:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4d0c4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1a4813a150 0x7f1a481330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.880 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a47fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a480974a0 0x7f1a48139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.881 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4d0c4700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1a4813a150 0x7f1a481330f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53848/0 (socket says 172.21.15.154:53848) 2022-11-27T17:01:59.881 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4d0c4700 1 -- 172.21.15.154:0/1894757378 learned_addr learned my addr 172.21.15.154:0/1894757378 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:01:59.881 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4c8c3700 1 --2- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a48139880 0x7f1a48099400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.881 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4d0c4700 1 -- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a48139880 msgr2=0x7f1a48099400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:01:59.882 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4d0c4700 1 --2- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a48139880 0x7f1a48099400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.882 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4d0c4700 1 -- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a480974a0 msgr2=0x7f1a48139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:01:59.882 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4d0c4700 1 --2- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a480974a0 0x7f1a48139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.882 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4d0c4700 1 -- 172.21.15.154:0/1894757378 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4805cb00 con 0x7f1a4813a150 2022-11-27T17:01:59.883 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a47fff700 1 --2- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a480974a0 0x7f1a48139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:01:59.883 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a4d0c4700 1 --2- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a4813a150 0x7f1a481330f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f1a400032f0 tx=0x7f1a40009290).ready entity=mon.0 client_cookie=a6a69febf002b65a server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:01:59.883 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a477fe700 1 -- 172.21.15.154:0/1894757378 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a4000b6c0 con 0x7f1a4813a150 2022-11-27T17:01:59.884 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a477fe700 1 -- 172.21.15.154:0/1894757378 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4000b820 con 0x7f1a4813a150 2022-11-27T17:01:59.884 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.869+0000 7f1a477fe700 1 -- 172.21.15.154:0/1894757378 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a4000b990 con 0x7f1a4813a150 2022-11-27T17:01:59.884 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.870+0000 7f1a4eb27700 1 -- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a4813a150 msgr2=0x7f1a481330f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:01:59.884 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.870+0000 7f1a4eb27700 1 --2- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a4813a150 0x7f1a481330f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f1a400032f0 tx=0x7f1a40009290).stop 2022-11-27T17:01:59.885 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.870+0000 7f1a4eb27700 1 -- 172.21.15.154:0/1894757378 shutdown_connections 2022-11-27T17:01:59.885 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.870+0000 7f1a4eb27700 1 --2- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a4813a150 0x7f1a481330f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.885 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.870+0000 7f1a4eb27700 1 --2- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a48139880 0x7f1a48099400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.885 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.870+0000 7f1a4eb27700 1 --2- 172.21.15.154:0/1894757378 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a480974a0 0x7f1a48139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.886 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.870+0000 7f1a4eb27700 1 -- 172.21.15.154:0/1894757378 >> 172.21.15.154:0/1894757378 conn(0x7f1a4805c4f0 msgr2=0x7f1a480a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:01:59.886 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.870+0000 7f1a4eb27700 1 -- 172.21.15.154:0/1894757378 shutdown_connections 2022-11-27T17:01:59.887 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.870+0000 7f1a4eb27700 1 -- 172.21.15.154:0/1894757378 wait complete. 2022-11-27T17:01:59.887 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.871+0000 7f1a4eb27700 1 Processor -- start 2022-11-27T17:01:59.887 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.871+0000 7f1a4eb27700 1 -- start start 2022-11-27T17:01:59.888 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4eb27700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a480974a0 0x7f1a48145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.888 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4eb27700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a48139880 0x7f1a48148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.889 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4eb27700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a4813a150 0x7f1a481490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:01:59.889 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4eb27700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480663a0 con 0x7f1a480974a0 2022-11-27T17:01:59.890 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4eb27700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48145fa0 con 0x7f1a48139880 2022-11-27T17:01:59.890 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4eb27700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48146240 con 0x7f1a4813a150 2022-11-27T17:01:59.890 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4c8c3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a480974a0 0x7f1a48145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.891 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4c8c3700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a480974a0 0x7f1a48145600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53858/0 (socket says 172.21.15.154:53858) 2022-11-27T17:01:59.891 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4c8c3700 1 -- 172.21.15.154:0/3856599546 learned_addr learned my addr 172.21.15.154:0/3856599546 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:01:59.891 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4d0c4700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a4813a150 0x7f1a481490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.892 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a47fff700 1 --2- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a48139880 0x7f1a48148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:01:59.892 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4c8c3700 1 -- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a4813a150 msgr2=0x7f1a481490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:01:59.893 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4c8c3700 1 --2- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a4813a150 0x7f1a481490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.893 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4c8c3700 1 -- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a48139880 msgr2=0x7f1a48148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:01:59.894 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4c8c3700 1 --2- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a48139880 0x7f1a48148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.894 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4c8c3700 1 -- 172.21.15.154:0/3856599546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3400b180 con 0x7f1a480974a0 2022-11-27T17:01:59.895 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4d0c4700 1 --2- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a4813a150 0x7f1a481490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:01:59.895 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.872+0000 7f1a4c8c3700 1 --2- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a480974a0 0x7f1a48145600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f1a3c00c000 tx=0x7f1a3c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:01:59.895 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.873+0000 7f1a45ffb700 1 -- 172.21.15.154:0/3856599546 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a3c010780 con 0x7f1a480974a0 2022-11-27T17:01:59.896 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.873+0000 7f1a45ffb700 1 -- 172.21.15.154:0/3856599546 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c0108e0 con 0x7f1a480974a0 2022-11-27T17:01:59.896 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.873+0000 7f1a4eb27700 1 -- 172.21.15.154:0/3856599546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a40009530 con 0x7f1a480974a0 2022-11-27T17:01:59.897 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.873+0000 7f1a45ffb700 1 -- 172.21.15.154:0/3856599546 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a3c010c40 con 0x7f1a480974a0 2022-11-27T17:01:59.897 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.873+0000 7f1a4eb27700 1 -- 172.21.15.154:0/3856599546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a48064450 con 0x7f1a480974a0 2022-11-27T17:01:59.898 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.873+0000 7f1a45ffb700 1 -- 172.21.15.154:0/3856599546 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f1a3c01c460 con 0x7f1a480974a0 2022-11-27T17:01:59.898 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.873+0000 7f1a45ffb700 1 -- 172.21.15.154:0/3856599546 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1a3c0123f0 con 0x7f1a480974a0 2022-11-27T17:01:59.898 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.874+0000 7f1a4eb27700 1 -- 172.21.15.154:0/3856599546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a48067120 con 0x7f1a480974a0 2022-11-27T17:01:59.900 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.876+0000 7f1a45ffb700 1 -- 172.21.15.154:0/3856599546 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f1a48067120 con 0x7f1a480974a0 2022-11-27T17:01:59.953 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.951+0000 7f1a4eb27700 1 -- 172.21.15.154:0/3856599546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3061801a-e670-4a13-9a7d-6c526f88ef28", "id": 0} v 0) v1 -- 0x7f1a48057a60 con 0x7f1a480974a0 2022-11-27T17:01:59.961 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.959+0000 7f1a45ffb700 1 -- 172.21.15.154:0/3856599546 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3061801a-e670-4a13-9a7d-6c526f88ef28", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1a3c0125b0 con 0x7f1a480974a0 2022-11-27T17:01:59.962 INFO:teuthology.orchestra.run.smithi154.stdout:0 2022-11-27T17:01:59.963 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.960+0000 7f1a4eb27700 1 -- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a480974a0 msgr2=0x7f1a48145600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:01:59.964 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.960+0000 7f1a4eb27700 1 --2- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a480974a0 0x7f1a48145600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f1a3c00c000 tx=0x7f1a3c006df0).stop 2022-11-27T17:01:59.964 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.960+0000 7f1a4eb27700 1 -- 172.21.15.154:0/3856599546 shutdown_connections 2022-11-27T17:01:59.964 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.960+0000 7f1a4eb27700 1 --2- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f1a4813a150 0x7f1a481490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.964 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.960+0000 7f1a4eb27700 1 --2- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1a480974a0 0x7f1a48145600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.965 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.960+0000 7f1a4eb27700 1 --2- 172.21.15.154:0/3856599546 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f1a48139880 0x7f1a48148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:01:59.965 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.960+0000 7f1a4eb27700 1 -- 172.21.15.154:0/3856599546 >> 172.21.15.154:0/3856599546 conn(0x7f1a4805c4f0 msgr2=0x7f1a48136bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:01:59.965 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.960+0000 7f1a4eb27700 1 -- 172.21.15.154:0/3856599546 shutdown_connections 2022-11-27T17:01:59.965 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:01:59.960+0000 7f1a4eb27700 1 -- 172.21.15.154:0/3856599546 wait complete. 2022-11-27T17:01:59.974 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 6906d403-7e5e-4ac7-9739-9178ecf56f2d 1 2022-11-27T17:02:00.083 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.074+0000 7fcbca776700 1 Processor -- start 2022-11-27T17:02:00.083 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.074+0000 7fcbca776700 1 -- start start 2022-11-27T17:02:00.084 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.075+0000 7fcbca776700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc4136820 0x7fcbc4136c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.084 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.075+0000 7fcbca776700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcbc40a5d40 0x7fcbc4140af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.084 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.075+0000 7fcbca776700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc40a4d90 con 0x7fcbc40a1250 2022-11-27T17:02:00.084 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.075+0000 7fcbca776700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc4060970 con 0x7fcbc40a5d40 2022-11-27T17:02:00.085 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.075+0000 7fcbca776700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc40a5030 con 0x7fcbc4136820 2022-11-27T17:02:00.085 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.075+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc4136820 0x7fcbc4136c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.085 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.075+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc4136820 0x7fcbc4136c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:54804/0 (socket says 172.21.15.154:54804) 2022-11-27T17:02:00.085 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.075+0000 7fcbc8d13700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcbc40a5d40 0x7fcbc4140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.085 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.075+0000 7fcbc3fff700 1 -- 172.21.15.154:0/284912526 learned_addr learned my addr 172.21.15.154:0/284912526 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:00.086 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.076+0000 7fcbc3fff700 1 -- 172.21.15.154:0/284912526 >> v2:172.21.15.154:3300/0 conn(0x7fcbc40a5d40 msgr2=0x7fcbc4140af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:00.086 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.076+0000 7fcbc3fff700 1 --2- 172.21.15.154:0/284912526 >> v2:172.21.15.154:3300/0 conn(0x7fcbc40a5d40 0x7fcbc4140af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.086 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.076+0000 7fcbc3fff700 1 -- 172.21.15.154:0/284912526 >> v1:172.21.15.154:6789/0 conn(0x7fcbc40a1250 legacy=0x7fcbc4060260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:00.086 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.076+0000 7fcbc3fff700 1 -- 172.21.15.154:0/284912526 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc4061d20 con 0x7fcbc4136820 2022-11-27T17:02:00.087 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.076+0000 7fcbc8d13700 1 --2- 172.21.15.154:0/284912526 >> v2:172.21.15.154:3300/0 conn(0x7fcbc40a5d40 0x7fcbc4140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:00.087 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.076+0000 7fcbc3fff700 1 --2- 172.21.15.154:0/284912526 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc4136820 0x7fcbc4136c00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fcbc405ced0 tx=0x7fcbb4006eb0).ready entity=mon.2 client_cookie=711efffc8ff5f0c4 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:00.087 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.076+0000 7fcbc2ffd700 1 -- 172.21.15.154:0/284912526 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbb400d570 con 0x7fcbc4136820 2022-11-27T17:02:00.088 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.076+0000 7fcbc2ffd700 1 -- 172.21.15.154:0/284912526 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb4000e90 con 0x7fcbc4136820 2022-11-27T17:02:00.088 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.076+0000 7fcbc2ffd700 1 -- 172.21.15.154:0/284912526 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbb400d8c0 con 0x7fcbc4136820 2022-11-27T17:02:00.088 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.077+0000 7fcbca776700 1 -- 172.21.15.154:0/284912526 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc4136820 msgr2=0x7fcbc4136c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.090 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.077+0000 7fcbca776700 1 --2- 172.21.15.154:0/284912526 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc4136820 0x7fcbc4136c00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fcbc405ced0 tx=0x7fcbb4006eb0).stop 2022-11-27T17:02:00.090 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.077+0000 7fcbca776700 1 -- 172.21.15.154:0/284912526 shutdown_connections 2022-11-27T17:02:00.091 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.077+0000 7fcbca776700 1 --2- 172.21.15.154:0/284912526 >> v2:172.21.15.154:3300/0 conn(0x7fcbc40a5d40 0x7fcbc4140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.091 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.077+0000 7fcbca776700 1 --2- 172.21.15.154:0/284912526 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc4136820 0x7fcbc4136c00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.091 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.077+0000 7fcbca776700 1 -- 172.21.15.154:0/284912526 >> 172.21.15.154:0/284912526 conn(0x7fcbc4062790 msgr2=0x7fcbc40a07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:00.091 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.077+0000 7fcbca776700 1 -- 172.21.15.154:0/284912526 shutdown_connections 2022-11-27T17:02:00.092 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.077+0000 7fcbca776700 1 -- 172.21.15.154:0/284912526 wait complete. 2022-11-27T17:02:00.092 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.078+0000 7fcbca776700 1 Processor -- start 2022-11-27T17:02:00.092 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.078+0000 7fcbca776700 1 -- start start 2022-11-27T17:02:00.092 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.078+0000 7fcbca776700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc40a1250 0x7fcbc41168d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.093 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.078+0000 7fcbca776700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc40a5d40 0x7fcbc4132830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.093 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.078+0000 7fcbca776700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbc4136820 0x7fcbc4132d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.093 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.078+0000 7fcbca776700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc4066db0 con 0x7fcbc40a1250 2022-11-27T17:02:00.093 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.078+0000 7fcbca776700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc40654e0 con 0x7fcbc4136820 2022-11-27T17:02:00.093 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.078+0000 7fcbca776700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc40672c0 con 0x7fcbc40a5d40 2022-11-27T17:02:00.094 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc40a1250 0x7fcbc41168d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.094 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc3fff700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc40a1250 0x7fcbc41168d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53886/0 (socket says 172.21.15.154:53886) 2022-11-27T17:02:00.094 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc3fff700 1 -- 172.21.15.154:0/3209242809 learned_addr learned my addr 172.21.15.154:0/3209242809 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:00.094 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc37fe700 1 --2- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc40a5d40 0x7fcbc4132830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.095 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc8d13700 1 --2- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbc4136820 0x7fcbc4132d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.095 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc37fe700 1 -- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbc4136820 msgr2=0x7fcbc4132d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.095 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc37fe700 1 --2- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbc4136820 0x7fcbc4132d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.095 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc37fe700 1 -- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc40a1250 msgr2=0x7fcbc41168d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.096 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc37fe700 1 --2- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc40a1250 0x7fcbc41168d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.096 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc37fe700 1 -- 172.21.15.154:0/3209242809 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc009470 con 0x7fcbc40a5d40 2022-11-27T17:02:00.096 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc3fff700 1 --2- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc40a1250 0x7fcbc41168d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:00.097 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc37fe700 1 --2- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc40a5d40 0x7fcbc4132830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fcbc4066ef0 tx=0x7fcbb8007b70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:00.097 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc17fa700 1 -- 172.21.15.154:0/3209242809 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbb8007030 con 0x7fcbc40a5d40 2022-11-27T17:02:00.097 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc17fa700 1 -- 172.21.15.154:0/3209242809 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb8007190 con 0x7fcbc40a5d40 2022-11-27T17:02:00.098 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbca776700 1 -- 172.21.15.154:0/3209242809 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb400b1e0 con 0x7fcbc40a5d40 2022-11-27T17:02:00.098 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbc17fa700 1 -- 172.21.15.154:0/3209242809 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcbb8006510 con 0x7fcbc40a5d40 2022-11-27T17:02:00.099 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.079+0000 7fcbca776700 1 -- 172.21.15.154:0/3209242809 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc4064700 con 0x7fcbc40a5d40 2022-11-27T17:02:00.099 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.080+0000 7fcbc17fa700 1 -- 172.21.15.154:0/3209242809 <== mon.2 v2:172.21.15.154:3302/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fcbb8010030 con 0x7fcbc40a5d40 2022-11-27T17:02:00.100 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.080+0000 7fcbc17fa700 1 -- 172.21.15.154:0/3209242809 <== mon.2 v2:172.21.15.154:3302/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fcbb801d440 con 0x7fcbc40a5d40 2022-11-27T17:02:00.101 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.081+0000 7fcbca776700 1 -- 172.21.15.154:0/3209242809 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc4116590 con 0x7fcbc40a5d40 2022-11-27T17:02:00.101 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.083+0000 7fcbc17fa700 1 -- 172.21.15.154:0/3209242809 <== mon.2 v2:172.21.15.154:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fcbb8006e10 con 0x7fcbc40a5d40 2022-11-27T17:02:00.161 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.158+0000 7fcbca776700 1 -- 172.21.15.154:0/3209242809 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "osd new", "uuid": "6906d403-7e5e-4ac7-9739-9178ecf56f2d", "id": 1} v 0) v1 -- 0x7fcbc4057a60 con 0x7fcbc40a5d40 2022-11-27T17:02:00.169 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.166+0000 7fcbc17fa700 1 -- 172.21.15.154:0/3209242809 <== mon.2 v2:172.21.15.154:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6906d403-7e5e-4ac7-9739-9178ecf56f2d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcbb801d600 con 0x7fcbc40a5d40 2022-11-27T17:02:00.170 INFO:teuthology.orchestra.run.smithi154.stdout:1 2022-11-27T17:02:00.171 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.167+0000 7fcbca776700 1 -- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc40a5d40 msgr2=0x7fcbc4132830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.171 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.167+0000 7fcbca776700 1 --2- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc40a5d40 0x7fcbc4132830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fcbc4066ef0 tx=0x7fcbb8007b70).stop 2022-11-27T17:02:00.172 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.167+0000 7fcbca776700 1 -- 172.21.15.154:0/3209242809 shutdown_connections 2022-11-27T17:02:00.172 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.167+0000 7fcbca776700 1 --2- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fcbc4136820 0x7fcbc4132d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.172 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.167+0000 7fcbca776700 1 --2- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcbc40a1250 0x7fcbc41168d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.172 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.167+0000 7fcbca776700 1 --2- 172.21.15.154:0/3209242809 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fcbc40a5d40 0x7fcbc4132830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.173 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.167+0000 7fcbca776700 1 -- 172.21.15.154:0/3209242809 >> 172.21.15.154:0/3209242809 conn(0x7fcbc4062790 msgr2=0x7fcbc413a190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:00.173 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.167+0000 7fcbca776700 1 -- 172.21.15.154:0/3209242809 shutdown_connections 2022-11-27T17:02:00.173 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.167+0000 7fcbca776700 1 -- 172.21.15.154:0/3209242809 wait complete. 2022-11-27T17:02:00.181 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new 5bcff406-28bc-465d-8c76-56626ab7936a 2 2022-11-27T17:02:00.293 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aeed3c700 1 Processor -- start 2022-11-27T17:02:00.293 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aeed3c700 1 -- start start 2022-11-27T17:02:00.293 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aeed3c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ae8099400 0x7f7ae8139c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.294 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aeed3c700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 0x7f7ae8140e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.294 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aeed3c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae8099bd0 con 0x7f7ae8139880 2022-11-27T17:02:00.294 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aeed3c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae8097f20 con 0x7f7ae8099400 2022-11-27T17:02:00.294 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aeed3c700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8097c80 con 0x7f7ae813a150 2022-11-27T17:02:00.295 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aed2d9700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 0x7f7ae8140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.295 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7ae7fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ae8099400 0x7f7ae8139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.295 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aed2d9700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 0x7f7ae8140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:54824/0 (socket says 172.21.15.154:54824) 2022-11-27T17:02:00.296 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aed2d9700 1 -- 172.21.15.154:0/326855314 learned_addr learned my addr 172.21.15.154:0/326855314 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:00.296 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.288+0000 7f7aecad8700 1 --1- 172.21.15.154:0/326855314 >> v1:172.21.15.154:6789/0 conn(0x7f7ae8139880 0x7f7ae80974a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59860/0 (socket says 172.21.15.154:59860) 2022-11-27T17:02:00.296 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.289+0000 7f7aed2d9700 1 -- 172.21.15.154:0/326855314 >> v2:172.21.15.154:3300/0 conn(0x7f7ae8099400 msgr2=0x7f7ae8139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:00.297 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.289+0000 7f7aed2d9700 1 --2- 172.21.15.154:0/326855314 >> v2:172.21.15.154:3300/0 conn(0x7f7ae8099400 0x7f7ae8139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.297 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.289+0000 7f7aed2d9700 1 -- 172.21.15.154:0/326855314 >> v1:172.21.15.154:6789/0 conn(0x7f7ae8139880 legacy=0x7f7ae80974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.297 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.289+0000 7f7aed2d9700 1 -- 172.21.15.154:0/326855314 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae805cb00 con 0x7f7ae813a150 2022-11-27T17:02:00.297 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.289+0000 7f7ae7fff700 1 --2- 172.21.15.154:0/326855314 >> v2:172.21.15.154:3300/0 conn(0x7f7ae8099400 0x7f7ae8139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:00.298 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.289+0000 7f7aed2d9700 1 --2- 172.21.15.154:0/326855314 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 0x7f7ae8140e50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f7ae80a9430 tx=0x7f7ae0009290).ready entity=mon.2 client_cookie=4276e7290eaa17a0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:00.298 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.289+0000 7f7ae77fe700 1 -- 172.21.15.154:0/326855314 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ae000b860 con 0x7f7ae813a150 2022-11-27T17:02:00.298 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.289+0000 7f7ae77fe700 1 -- 172.21.15.154:0/326855314 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae000b9c0 con 0x7f7ae813a150 2022-11-27T17:02:00.298 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.289+0000 7f7ae77fe700 1 -- 172.21.15.154:0/326855314 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ae000bb30 con 0x7f7ae813a150 2022-11-27T17:02:00.298 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.290+0000 7f7aeed3c700 1 -- 172.21.15.154:0/326855314 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 msgr2=0x7f7ae8140e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.299 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.290+0000 7f7aeed3c700 1 --2- 172.21.15.154:0/326855314 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 0x7f7ae8140e50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f7ae80a9430 tx=0x7f7ae0009290).stop 2022-11-27T17:02:00.299 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.290+0000 7f7aeed3c700 1 -- 172.21.15.154:0/326855314 shutdown_connections 2022-11-27T17:02:00.299 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.290+0000 7f7aeed3c700 1 --2- 172.21.15.154:0/326855314 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 0x7f7ae8140e50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.299 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.290+0000 7f7aeed3c700 1 --2- 172.21.15.154:0/326855314 >> v2:172.21.15.154:3300/0 conn(0x7f7ae8099400 0x7f7ae8139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.300 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.290+0000 7f7aeed3c700 1 -- 172.21.15.154:0/326855314 >> 172.21.15.154:0/326855314 conn(0x7f7ae805c4f0 msgr2=0x7f7ae80a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:00.300 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.290+0000 7f7aeed3c700 1 -- 172.21.15.154:0/326855314 shutdown_connections 2022-11-27T17:02:00.300 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.290+0000 7f7aeed3c700 1 -- 172.21.15.154:0/326855314 wait complete. 2022-11-27T17:02:00.303 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.291+0000 7f7aeed3c700 1 Processor -- start 2022-11-27T17:02:00.304 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.291+0000 7f7aeed3c700 1 -- start start 2022-11-27T17:02:00.304 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.291+0000 7f7aeed3c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae8099400 0x7f7ae80653c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.304 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.291+0000 7f7aeed3c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae8139880 0x7f7ae8145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.304 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.291+0000 7f7aeed3c700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 0x7f7ae8146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.304 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.291+0000 7f7aeed3c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80972c0 con 0x7f7ae8099400 2022-11-27T17:02:00.305 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.291+0000 7f7aeed3c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8065d90 con 0x7f7ae8139880 2022-11-27T17:02:00.305 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.291+0000 7f7aeed3c700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80644f0 con 0x7f7ae813a150 2022-11-27T17:02:00.305 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.291+0000 7f7ae7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae8139880 0x7f7ae8145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.305 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.291+0000 7f7aecad8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae8099400 0x7f7ae80653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.306 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7aecad8700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae8099400 0x7f7ae80653c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53892/0 (socket says 172.21.15.154:53892) 2022-11-27T17:02:00.306 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7ae7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae8139880 0x7f7ae8145e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:60796/0 (socket says 172.21.15.154:60796) 2022-11-27T17:02:00.306 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7aecad8700 1 -- 172.21.15.154:0/2293731890 learned_addr learned my addr 172.21.15.154:0/2293731890 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:00.306 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7aed2d9700 1 --2- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 0x7f7ae8146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.307 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7ae7fff700 1 -- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 msgr2=0x7f7ae8146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.307 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7ae7fff700 1 --2- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 0x7f7ae8146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.307 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7ae7fff700 1 -- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae8099400 msgr2=0x7f7ae80653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.307 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7ae7fff700 1 --2- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae8099400 0x7f7ae80653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.308 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7ae7fff700 1 -- 172.21.15.154:0/2293731890 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad400b180 con 0x7f7ae8139880 2022-11-27T17:02:00.308 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7aecad8700 1 --2- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae8099400 0x7f7ae80653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:00.308 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7ae7fff700 1 --2- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae8139880 0x7f7ae8145e80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f7ae8063d30 tx=0x7f7ad4006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:00.308 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7ae5ffb700 1 -- 172.21.15.154:0/2293731890 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ad4010780 con 0x7f7ae8139880 2022-11-27T17:02:00.308 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7ae5ffb700 1 -- 172.21.15.154:0/2293731890 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad40108e0 con 0x7f7ae8139880 2022-11-27T17:02:00.309 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7ae5ffb700 1 -- 172.21.15.154:0/2293731890 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ad4010c40 con 0x7f7ae8139880 2022-11-27T17:02:00.309 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7aeed3c700 1 -- 172.21.15.154:0/2293731890 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae0009530 con 0x7f7ae8139880 2022-11-27T17:02:00.309 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.292+0000 7f7aeed3c700 1 -- 172.21.15.154:0/2293731890 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae8146860 con 0x7f7ae8139880 2022-11-27T17:02:00.310 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.293+0000 7f7ae5ffb700 1 -- 172.21.15.154:0/2293731890 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f7ad401c460 con 0x7f7ae8139880 2022-11-27T17:02:00.310 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.293+0000 7f7ae5ffb700 1 -- 172.21.15.154:0/2293731890 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f7ad40123f0 con 0x7f7ae8139880 2022-11-27T17:02:00.310 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.293+0000 7f7aeed3c700 1 -- 172.21.15.154:0/2293731890 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae8098c90 con 0x7f7ae8139880 2022-11-27T17:02:00.310 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.295+0000 7f7ae5ffb700 1 -- 172.21.15.154:0/2293731890 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f7ad401cba0 con 0x7f7ae8139880 2022-11-27T17:02:00.381 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.378+0000 7f7aeed3c700 1 -- 172.21.15.154:0/2293731890 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5bcff406-28bc-465d-8c76-56626ab7936a", "id": 2} v 0) v1 -- 0x7f7ae8057a60 con 0x7f7ae8139880 2022-11-27T17:02:00.384 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.381+0000 7f7ae5ffb700 1 -- 172.21.15.154:0/2293731890 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5bcff406-28bc-465d-8c76-56626ab7936a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7ad401cd80 con 0x7f7ae8139880 2022-11-27T17:02:00.384 INFO:teuthology.orchestra.run.smithi154.stdout:2 2022-11-27T17:02:00.385 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.382+0000 7f7aeed3c700 1 -- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae8139880 msgr2=0x7f7ae8145e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.386 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.382+0000 7f7aeed3c700 1 --2- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae8139880 0x7f7ae8145e80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f7ae8063d30 tx=0x7f7ad4006df0).stop 2022-11-27T17:02:00.386 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.382+0000 7f7aeed3c700 1 -- 172.21.15.154:0/2293731890 shutdown_connections 2022-11-27T17:02:00.386 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.382+0000 7f7aeed3c700 1 --2- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f7ae813a150 0x7f7ae8146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.386 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.382+0000 7f7aeed3c700 1 --2- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae8099400 0x7f7ae80653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.387 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.382+0000 7f7aeed3c700 1 --2- 172.21.15.154:0/2293731890 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7ae8139880 0x7f7ae8145e80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.387 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.382+0000 7f7aeed3c700 1 -- 172.21.15.154:0/2293731890 >> 172.21.15.154:0/2293731890 conn(0x7f7ae805c4f0 msgr2=0x7f7ae8137f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:00.387 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.382+0000 7f7aeed3c700 1 -- 172.21.15.154:0/2293731890 shutdown_connections 2022-11-27T17:02:00.387 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.382+0000 7f7aeed3c700 1 -- 172.21.15.154:0/2293731890 wait complete. 2022-11-27T17:02:00.395 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd new d6f7e6fc-d825-4e61-b433-6d5cf4075863 3 2022-11-27T17:02:00.511 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.497+0000 7f46ccebb700 1 Processor -- start 2022-11-27T17:02:00.511 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.498+0000 7f46ccebb700 1 -- start start 2022-11-27T17:02:00.512 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.498+0000 7f46ccebb700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 0x7f46c809a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.512 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.498+0000 7f46ccebb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c81339a0 0x7f46c81369b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.513 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.498+0000 7f46ccebb700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c8099920 con 0x7f46c8133580 2022-11-27T17:02:00.513 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.498+0000 7f46ccebb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8099be0 con 0x7f46c81339a0 2022-11-27T17:02:00.514 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.498+0000 7f46ccebb700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46c809a900 con 0x7f46c809a030 2022-11-27T17:02:00.514 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.498+0000 7f46c659c700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 0x7f46c809a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.515 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.498+0000 7f46c5d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f46c8133580 0x7f46c81362a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59874/0 (socket says 172.21.15.154:59874) 2022-11-27T17:02:00.515 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.498+0000 7f46c5d9b700 1 -- 172.21.15.154:0/602040906 learned_addr learned my addr 172.21.15.154:0/602040906 (peer_addr_for_me v1:172.21.15.154:0/0) 2022-11-27T17:02:00.515 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c6d9d700 1 --2- 172.21.15.154:0/602040906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c81339a0 0x7f46c81369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.516 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c559a700 1 -- 172.21.15.154:0/602040906 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407850000 0 0) 0x7f46c8099920 con 0x7f46c8133580 2022-11-27T17:02:00.516 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c559a700 1 -- 172.21.15.154:0/602040906 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ac003260 con 0x7f46c8133580 2022-11-27T17:02:00.517 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c659c700 1 -- 172.21.15.154:0/602040906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c81339a0 msgr2=0x7f46c81369b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:00.517 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c659c700 1 --2- 172.21.15.154:0/602040906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c81339a0 0x7f46c81369b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.518 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c659c700 1 -- 172.21.15.154:0/602040906 >> v1:172.21.15.154:6789/0 conn(0x7f46c8133580 legacy=0x7f46c81362a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.518 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c659c700 1 -- 172.21.15.154:0/602040906 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c809b7b0 con 0x7f46c809a030 2022-11-27T17:02:00.519 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c659c700 1 --2- 172.21.15.154:0/602040906 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 0x7f46c809a410 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f46b8002f70 tx=0x7f46b8006df0).ready entity=mon.2 client_cookie=5358921c6a602dd4 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:00.519 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c559a700 1 -- 172.21.15.154:0/602040906 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46b800d570 con 0x7f46c809a030 2022-11-27T17:02:00.520 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c559a700 1 -- 172.21.15.154:0/602040906 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b8000e90 con 0x7f46c809a030 2022-11-27T17:02:00.520 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.499+0000 7f46c559a700 1 -- 172.21.15.154:0/602040906 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46b800d8c0 con 0x7f46c809a030 2022-11-27T17:02:00.520 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.500+0000 7f46ccebb700 1 -- 172.21.15.154:0/602040906 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 msgr2=0x7f46c809a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.521 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.500+0000 7f46ccebb700 1 --2- 172.21.15.154:0/602040906 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 0x7f46c809a410 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f46b8002f70 tx=0x7f46b8006df0).stop 2022-11-27T17:02:00.521 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.500+0000 7f46ccebb700 1 -- 172.21.15.154:0/602040906 shutdown_connections 2022-11-27T17:02:00.522 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.500+0000 7f46ccebb700 1 --2- 172.21.15.154:0/602040906 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c81339a0 0x7f46c81369b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.523 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.500+0000 7f46ccebb700 1 --2- 172.21.15.154:0/602040906 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 0x7f46c809a410 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.523 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.500+0000 7f46ccebb700 1 -- 172.21.15.154:0/602040906 >> 172.21.15.154:0/602040906 conn(0x7f46c80616e0 msgr2=0x7f46c805cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:00.523 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.500+0000 7f46ccebb700 1 -- 172.21.15.154:0/602040906 shutdown_connections 2022-11-27T17:02:00.524 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.500+0000 7f46ccebb700 1 -- 172.21.15.154:0/602040906 wait complete. 2022-11-27T17:02:00.524 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.501+0000 7f46ccebb700 1 Processor -- start 2022-11-27T17:02:00.524 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.501+0000 7f46ccebb700 1 -- start start 2022-11-27T17:02:00.524 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.501+0000 7f46ccebb700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 0x7f46c8065000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.525 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.501+0000 7f46ccebb700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c8133580 0x7f46c812fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.525 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.501+0000 7f46ccebb700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46c81339a0 0x7f46c8130230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.525 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.501+0000 7f46ccebb700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80926d0 con 0x7f46c81339a0 2022-11-27T17:02:00.525 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.501+0000 7f46ccebb700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8093030 con 0x7f46c8133580 2022-11-27T17:02:00.525 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.501+0000 7f46ccebb700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8065990 con 0x7f46c809a030 2022-11-27T17:02:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c5d9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c8133580 0x7f46c812fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c5d9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c8133580 0x7f46c812fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:60824/0 (socket says 172.21.15.154:60824) 2022-11-27T17:02:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c5d9b700 1 -- 172.21.15.154:0/2093137507 learned_addr learned my addr 172.21.15.154:0/2093137507 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c6d9d700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46c81339a0 0x7f46c8130230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c659c700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 0x7f46c8065000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.527 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c5d9b700 1 -- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 msgr2=0x7f46c8065000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.527 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c5d9b700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 0x7f46c8065000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.527 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c5d9b700 1 -- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46c81339a0 msgr2=0x7f46c8130230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.527 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c5d9b700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46c81339a0 0x7f46c8130230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.528 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c5d9b700 1 -- 172.21.15.154:0/2093137507 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b800b180 con 0x7f46c8133580 2022-11-27T17:02:00.528 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c659c700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 0x7f46c8065000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T17:02:00.528 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c6d9d700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46c81339a0 0x7f46c8130230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:00.528 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46c5d9b700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c8133580 0x7f46c812fd40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f46bc001110 tx=0x7f46bc007aa0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:00.528 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46b77fe700 1 -- 172.21.15.154:0/2093137507 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46bc0070d0 con 0x7f46c8133580 2022-11-27T17:02:00.529 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46b77fe700 1 -- 172.21.15.154:0/2093137507 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc007230 con 0x7f46c8133580 2022-11-27T17:02:00.529 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46b77fe700 1 -- 172.21.15.154:0/2093137507 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46bc005610 con 0x7f46c8133580 2022-11-27T17:02:00.529 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46ccebb700 1 -- 172.21.15.154:0/2093137507 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c8065c50 con 0x7f46c8133580 2022-11-27T17:02:00.529 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46ccebb700 1 -- 172.21.15.154:0/2093137507 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c8091d50 con 0x7f46c8133580 2022-11-27T17:02:00.530 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.502+0000 7f46b77fe700 1 -- 172.21.15.154:0/2093137507 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f46bc010070 con 0x7f46c8133580 2022-11-27T17:02:00.530 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.503+0000 7f46b77fe700 1 -- 172.21.15.154:0/2093137507 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f46bc005940 con 0x7f46c8133580 2022-11-27T17:02:00.531 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.503+0000 7f46ccebb700 1 -- 172.21.15.154:0/2093137507 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c8093170 con 0x7f46c8133580 2022-11-27T17:02:00.531 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.507+0000 7f46b77fe700 1 -- 172.21.15.154:0/2093137507 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f46bc005da0 con 0x7f46c8133580 2022-11-27T17:02:00.585 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.582+0000 7f46ccebb700 1 -- 172.21.15.154:0/2093137507 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d6f7e6fc-d825-4e61-b433-6d5cf4075863", "id": 3} v 0) v1 -- 0x7f46c8066bc0 con 0x7f46c8133580 2022-11-27T17:02:00.588 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.585+0000 7f46b77fe700 1 -- 172.21.15.154:0/2093137507 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d6f7e6fc-d825-4e61-b433-6d5cf4075863", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f46bc013450 con 0x7f46c8133580 2022-11-27T17:02:00.588 INFO:teuthology.orchestra.run.smithi154.stdout:3 2022-11-27T17:02:00.589 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.586+0000 7f46ccebb700 1 -- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c8133580 msgr2=0x7f46c812fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.590 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.586+0000 7f46ccebb700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c8133580 0x7f46c812fd40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f46bc001110 tx=0x7f46bc007aa0).stop 2022-11-27T17:02:00.590 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.586+0000 7f46ccebb700 1 -- 172.21.15.154:0/2093137507 shutdown_connections 2022-11-27T17:02:00.590 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.586+0000 7f46ccebb700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46c81339a0 0x7f46c8130230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.590 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.586+0000 7f46ccebb700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f46c809a030 0x7f46c8065000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.591 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.586+0000 7f46ccebb700 1 --2- 172.21.15.154:0/2093137507 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f46c8133580 0x7f46c812fd40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.591 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.586+0000 7f46ccebb700 1 -- 172.21.15.154:0/2093137507 >> 172.21.15.154:0/2093137507 conn(0x7f46c80616e0 msgr2=0x7f46c81363b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:00.591 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.587+0000 7f46ccebb700 1 -- 172.21.15.154:0/2093137507 shutdown_connections 2022-11-27T17:02:00.591 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.587+0000 7f46ccebb700 1 -- 172.21.15.154:0/2093137507 wait complete. 2022-11-27T17:02:00.602 INFO:tasks.ceph.osd.0:Restarting daemon 2022-11-27T17:02:00.602 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-11-27T17:02:00.605 INFO:tasks.ceph.osd.0:Started 2022-11-27T17:02:00.605 INFO:tasks.ceph.osd.1:Restarting daemon 2022-11-27T17:02:00.606 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-11-27T17:02:00.608 INFO:tasks.ceph.osd.1:Started 2022-11-27T17:02:00.609 INFO:tasks.ceph.osd.2:Restarting daemon 2022-11-27T17:02:00.609 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-11-27T17:02:00.612 INFO:tasks.ceph.osd.2:Started 2022-11-27T17:02:00.612 INFO:tasks.ceph.osd.3:Restarting daemon 2022-11-27T17:02:00.613 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-11-27T17:02:00.615 INFO:tasks.ceph.osd.3:Started 2022-11-27T17:02:00.616 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T17:02:00.778 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.766+0000 7f27cf2e9700 1 Processor -- start 2022-11-27T17:02:00.778 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.766+0000 7f27cf2e9700 1 -- start start 2022-11-27T17:02:00.779 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cf2e9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c81368f0 0x7f27c80669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.779 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cf2e9700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c8064b00 0x7f27c80653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.779 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cf2e9700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c805ccd0 con 0x7f27c8136510 2022-11-27T17:02:00.780 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cf2e9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8064fb0 con 0x7f27c81368f0 2022-11-27T17:02:00.780 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cf2e9700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8136030 con 0x7f27c8064b00 2022-11-27T17:02:00.781 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cd085700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f27c8136510 0x7f27c8062a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59912/0 (socket says 172.21.15.154:59912) 2022-11-27T17:02:00.781 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cc884700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c81368f0 0x7f27c80669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.781 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cd085700 1 -- 172.21.15.154:0/4267440479 learned_addr learned my addr 172.21.15.154:0/4267440479 (peer_addr_for_me v1:172.21.15.154:0/0) 2022-11-27T17:02:00.782 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cd886700 1 --2- 172.21.15.154:0/4267440479 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c8064b00 0x7f27c80653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.782 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cd886700 1 -- 172.21.15.154:0/4267440479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c81368f0 msgr2=0x7f27c80669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:00.783 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27b7fff700 1 -- 172.21.15.154:0/4267440479 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476895400 0 0) 0x7f27c805ccd0 con 0x7f27c8136510 2022-11-27T17:02:00.783 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cd886700 1 --2- 172.21.15.154:0/4267440479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c81368f0 0x7f27c80669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.783 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cd886700 1 -- 172.21.15.154:0/4267440479 >> v1:172.21.15.154:6789/0 conn(0x7f27c8136510 legacy=0x7f27c8062a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.784 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cd886700 1 -- 172.21.15.154:0/4267440479 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c81362f0 con 0x7f27c8064b00 2022-11-27T17:02:00.784 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.767+0000 7f27cc884700 1 --2- 172.21.15.154:0/4267440479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c81368f0 0x7f27c80669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:00.785 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.768+0000 7f27cd886700 1 --2- 172.21.15.154:0/4267440479 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c8064b00 0x7f27c80653e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f27c00032f0 tx=0x7f27c000b470).ready entity=mon.2 client_cookie=b4d07bfb73b84fdc server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:00.785 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.768+0000 7f27b7fff700 1 -- 172.21.15.154:0/4267440479 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27c000dd70 con 0x7f27c8064b00 2022-11-27T17:02:00.785 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.768+0000 7f27b7fff700 1 -- 172.21.15.154:0/4267440479 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c0010470 con 0x7f27c8064b00 2022-11-27T17:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.768+0000 7f27b7fff700 1 -- 172.21.15.154:0/4267440479 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27c00107d0 con 0x7f27c8064b00 2022-11-27T17:02:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.768+0000 7f27cf2e9700 1 -- 172.21.15.154:0/4267440479 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c8064b00 msgr2=0x7f27c80653e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.768+0000 7f27cf2e9700 1 --2- 172.21.15.154:0/4267440479 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c8064b00 0x7f27c80653e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f27c00032f0 tx=0x7f27c000b470).stop 2022-11-27T17:02:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.768+0000 7f27cf2e9700 1 -- 172.21.15.154:0/4267440479 shutdown_connections 2022-11-27T17:02:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.768+0000 7f27cf2e9700 1 --2- 172.21.15.154:0/4267440479 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c8064b00 0x7f27c80653e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.788 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.768+0000 7f27cf2e9700 1 --2- 172.21.15.154:0/4267440479 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c81368f0 0x7f27c80669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.788 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.768+0000 7f27cf2e9700 1 -- 172.21.15.154:0/4267440479 >> 172.21.15.154:0/4267440479 conn(0x7f27c805c4f0 msgr2=0x7f27c80a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.769+0000 7f27cf2e9700 1 -- 172.21.15.154:0/4267440479 shutdown_connections 2022-11-27T17:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.769+0000 7f27cf2e9700 1 -- 172.21.15.154:0/4267440479 wait complete. 2022-11-27T17:02:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.770+0000 7f27cf2e9700 1 Processor -- start 2022-11-27T17:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.770+0000 7f27cf2e9700 1 -- start start 2022-11-27T17:02:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cf2e9700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8064b00 0x7f27c8097f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cf2e9700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c8136510 0x7f27c8098fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cf2e9700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c81368f0 0x7f27c8146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:00.791 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cf2e9700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8097af0 con 0x7f27c8064b00 2022-11-27T17:02:00.792 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cf2e9700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81439d0 con 0x7f27c8136510 2022-11-27T17:02:00.792 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cf2e9700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8133520 con 0x7f27c81368f0 2022-11-27T17:02:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cc884700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c8136510 0x7f27c8098fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cd886700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c81368f0 0x7f27c8146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cc884700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c8136510 0x7f27c8098fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:60882/0 (socket says 172.21.15.154:60882) 2022-11-27T17:02:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cd886700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c81368f0 0x7f27c8146470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:54894/0 (socket says 172.21.15.154:54894) 2022-11-27T17:02:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cc884700 1 -- 172.21.15.154:0/2132959738 learned_addr learned my addr 172.21.15.154:0/2132959738 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cd085700 1 --2- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8064b00 0x7f27c8097f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cc884700 1 -- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c81368f0 msgr2=0x7f27c8146470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.796 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cc884700 1 --2- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c81368f0 0x7f27c8146470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.796 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cc884700 1 -- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8064b00 msgr2=0x7f27c8097f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.797 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cc884700 1 --2- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8064b00 0x7f27c8097f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.797 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cc884700 1 -- 172.21.15.154:0/2132959738 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b800b180 con 0x7f27c8136510 2022-11-27T17:02:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.771+0000 7f27cd886700 1 --2- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c81368f0 0x7f27c8146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.772+0000 7f27cc884700 1 --2- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c8136510 0x7f27c8098fb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f27c8060850 tx=0x7f27b8006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.772+0000 7f27b67fc700 1 -- 172.21.15.154:0/2132959738 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27b8010780 con 0x7f27c8136510 2022-11-27T17:02:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.772+0000 7f27b67fc700 1 -- 172.21.15.154:0/2132959738 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b80108e0 con 0x7f27c8136510 2022-11-27T17:02:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.772+0000 7f27cf2e9700 1 -- 172.21.15.154:0/2132959738 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c000b710 con 0x7f27c8136510 2022-11-27T17:02:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.772+0000 7f27b67fc700 1 -- 172.21.15.154:0/2132959738 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27b8010c40 con 0x7f27c8136510 2022-11-27T17:02:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.772+0000 7f27b67fc700 1 -- 172.21.15.154:0/2132959738 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f27b801c460 con 0x7f27c8136510 2022-11-27T17:02:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.772+0000 7f27cf2e9700 1 -- 172.21.15.154:0/2132959738 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c80994a0 con 0x7f27c8136510 2022-11-27T17:02:00.801 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.772+0000 7f27b67fc700 1 -- 172.21.15.154:0/2132959738 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f27b801cbf0 con 0x7f27c8136510 2022-11-27T17:02:00.801 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.773+0000 7f27cf2e9700 1 -- 172.21.15.154:0/2132959738 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c80988d0 con 0x7f27c8136510 2022-11-27T17:02:00.802 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.777+0000 7f27b67fc700 1 -- 172.21.15.154:0/2132959738 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f27b801cdb0 con 0x7f27c8136510 2022-11-27T17:02:00.844 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.840+0000 7f27cf2e9700 1 -- 172.21.15.154:0/2132959738 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27c8135c30 con 0x7f27c8136510 2022-11-27T17:02:00.845 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.841+0000 7f27b67fc700 1 -- 172.21.15.154:0/2132959738 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+3913 (secure 0 0 0) 0x7f27b8026430 con 0x7f27c8136510 2022-11-27T17:02:00.846 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:02:00.846 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":6,"fsid":"5adefcc8-72cd-433b-bd69-ec1eac0a6bf2","created":"2022-11-27T17:01:59.370943+0000","modified":"2022-11-27T17:02:00.584387+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"3061801a-e670-4a13-9a7d-6c526f88ef28","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6906d403-7e5e-4ac7-9739-9178ecf56f2d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5bcff406-28bc-465d-8c76-56626ab7936a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d6f7e6fc-d825-4e61-b433-6d5cf4075863","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T17:02:00.848 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.842+0000 7f27cf2e9700 1 -- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c8136510 msgr2=0x7f27c8098fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:00.848 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.842+0000 7f27cf2e9700 1 --2- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c8136510 0x7f27c8098fb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f27c8060850 tx=0x7f27b8006df0).stop 2022-11-27T17:02:00.849 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.842+0000 7f27cf2e9700 1 -- 172.21.15.154:0/2132959738 shutdown_connections 2022-11-27T17:02:00.849 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.842+0000 7f27cf2e9700 1 --2- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f27c81368f0 0x7f27c8146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.849 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.842+0000 7f27cf2e9700 1 --2- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8064b00 0x7f27c8097f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.850 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.842+0000 7f27cf2e9700 1 --2- 172.21.15.154:0/2132959738 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f27c8136510 0x7f27c8098fb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:00.850 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.842+0000 7f27cf2e9700 1 -- 172.21.15.154:0/2132959738 >> 172.21.15.154:0/2132959738 conn(0x7f27c805c4f0 msgr2=0x7f27c8136f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:00.850 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.842+0000 7f27cf2e9700 1 -- 172.21.15.154:0/2132959738 shutdown_connections 2022-11-27T17:02:00.851 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:00.843+0000 7f27cf2e9700 1 -- 172.21.15.154:0/2132959738 wait complete. 2022-11-27T17:02:00.856 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-11-27T17:02:00.857 INFO:tasks.ceph:Waiting for OSDs to come up 2022-11-27T17:02:01.246 INFO:tasks.ceph.osd.1.smithi154.stderr:2022-11-27T17:02:01.242+0000 7f61a51a2f00 -1 Falling back to public interface 2022-11-27T17:02:01.246 INFO:tasks.ceph.osd.2.smithi154.stderr:2022-11-27T17:02:01.243+0000 7f1308056f00 -1 Falling back to public interface 2022-11-27T17:02:01.251 INFO:tasks.ceph.osd.0.smithi154.stderr:2022-11-27T17:02:01.248+0000 7f28a8f57f00 -1 Falling back to public interface 2022-11-27T17:02:01.255 INFO:tasks.ceph.osd.3.smithi154.stderr:2022-11-27T17:02:01.252+0000 7f15b93c0f00 -1 Falling back to public interface 2022-11-27T17:02:01.262 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T17:02:01.367 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36bb0d6700 1 Processor -- start 2022-11-27T17:02:01.367 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36bb0d6700 1 -- start start 2022-11-27T17:02:01.368 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36bb0d6700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b40973a0 0x7f36b4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:01.368 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36bb0d6700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36b4097780 0x7f36b409a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:01.368 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36bb0d6700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b4098de0 con 0x7f36b40992e0 2022-11-27T17:02:01.368 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36bb0d6700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b4098be0 con 0x7f36b4097780 2022-11-27T17:02:01.369 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36bb0d6700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36b409a960 con 0x7f36b40973a0 2022-11-27T17:02:01.369 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36b3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36b4097780 0x7f36b409a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:01.369 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36b8e72700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b40973a0 0x7f36b4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:01.370 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36b8e72700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b40973a0 0x7f36b4099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:54962/0 (socket says 172.21.15.154:54962) 2022-11-27T17:02:01.370 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36b3fff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36b4097780 0x7f36b409a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53996/0 (socket says 172.21.15.154:53996) 2022-11-27T17:02:01.370 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36b8e72700 1 -- 172.21.15.154:0/4119975482 learned_addr learned my addr 172.21.15.154:0/4119975482 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:01.370 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.362+0000 7f36b9673700 1 --1- 172.21.15.154:0/4119975482 >> v1:172.21.15.154:6789/0 conn(0x7f36b40992e0 0x7f36b4139610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59928/0 (socket says 172.21.15.154:59928) 2022-11-27T17:02:01.370 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36b3fff700 1 -- 172.21.15.154:0/4119975482 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b40973a0 msgr2=0x7f36b4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:01.371 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36b3fff700 1 --2- 172.21.15.154:0/4119975482 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b40973a0 0x7f36b4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:01.371 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36b3fff700 1 -- 172.21.15.154:0/4119975482 >> v1:172.21.15.154:6789/0 conn(0x7f36b40992e0 legacy=0x7f36b4139610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:01.371 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36b3fff700 1 -- 172.21.15.154:0/4119975482 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b409ac20 con 0x7f36b4097780 2022-11-27T17:02:01.372 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36b37fe700 1 -- 172.21.15.154:0/4119975482 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178858440 0 0) 0x7f36b4098de0 con 0x7f36b40992e0 2022-11-27T17:02:01.372 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36b8e72700 1 --2- 172.21.15.154:0/4119975482 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b40973a0 0x7f36b4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:01.372 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36b3fff700 1 --2- 172.21.15.154:0/4119975482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b4097780 0x7f36b409a360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f369c002f70 tx=0x7f369c006df0).ready entity=mon.0 client_cookie=7740e483adb35cba server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:01.373 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36b37fe700 1 -- 172.21.15.154:0/4119975482 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f369c00d570 con 0x7f36b4097780 2022-11-27T17:02:01.373 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36b37fe700 1 -- 172.21.15.154:0/4119975482 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c000e90 con 0x7f36b4097780 2022-11-27T17:02:01.374 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36b37fe700 1 -- 172.21.15.154:0/4119975482 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f369c00d8c0 con 0x7f36b4097780 2022-11-27T17:02:01.374 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36bb0d6700 1 -- 172.21.15.154:0/4119975482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b4097780 msgr2=0x7f36b409a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:01.374 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.363+0000 7f36bb0d6700 1 --2- 172.21.15.154:0/4119975482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b4097780 0x7f36b409a360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f369c002f70 tx=0x7f369c006df0).stop 2022-11-27T17:02:01.374 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.364+0000 7f36bb0d6700 1 -- 172.21.15.154:0/4119975482 shutdown_connections 2022-11-27T17:02:01.375 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.364+0000 7f36bb0d6700 1 --2- 172.21.15.154:0/4119975482 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b40973a0 0x7f36b4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:01.375 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.364+0000 7f36bb0d6700 1 --2- 172.21.15.154:0/4119975482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b4097780 0x7f36b409a360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:01.375 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.364+0000 7f36bb0d6700 1 -- 172.21.15.154:0/4119975482 >> 172.21.15.154:0/4119975482 conn(0x7f36b405c4f0 msgr2=0x7f36b40a89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:01.375 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.364+0000 7f36bb0d6700 1 -- 172.21.15.154:0/4119975482 shutdown_connections 2022-11-27T17:02:01.376 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.364+0000 7f36bb0d6700 1 -- 172.21.15.154:0/4119975482 wait complete. 2022-11-27T17:02:01.376 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.364+0000 7f36bb0d6700 1 Processor -- start 2022-11-27T17:02:01.376 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.364+0000 7f36bb0d6700 1 -- start start 2022-11-27T17:02:01.379 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36bb0d6700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b40973a0 0x7f36b409ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:01.379 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36bb0d6700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b4097780 0x7f36b4148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:01.380 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36bb0d6700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b40992e0 0x7f36b41492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:01.380 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36bb0d6700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40673a0 con 0x7f36b40973a0 2022-11-27T17:02:01.380 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36bb0d6700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b409d3f0 con 0x7f36b40992e0 2022-11-27T17:02:01.380 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36bb0d6700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36b409f320 con 0x7f36b4097780 2022-11-27T17:02:01.380 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36b8e72700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b40973a0 0x7f36b409ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:01.381 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36b3fff700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b4097780 0x7f36b4148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:01.381 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36b8e72700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b40973a0 0x7f36b409ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54000/0 (socket says 172.21.15.154:54000) 2022-11-27T17:02:01.381 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36b3fff700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b4097780 0x7f36b4148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:54972/0 (socket says 172.21.15.154:54972) 2022-11-27T17:02:01.382 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36b8e72700 1 -- 172.21.15.154:0/478910465 learned_addr learned my addr 172.21.15.154:0/478910465 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:01.382 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36b9673700 1 --2- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b40992e0 0x7f36b41492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:01.383 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36b8e72700 1 -- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b4097780 msgr2=0x7f36b4148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:01.383 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.365+0000 7f36b8e72700 1 --2- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b4097780 0x7f36b4148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:01.383 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36b8e72700 1 -- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b40992e0 msgr2=0x7f36b41492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:01.383 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36b8e72700 1 --2- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b40992e0 0x7f36b41492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:01.384 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36b8e72700 1 -- 172.21.15.154:0/478910465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac009530 con 0x7f36b40973a0 2022-11-27T17:02:01.384 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36b3fff700 1 --2- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b4097780 0x7f36b4148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:01.385 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36b9673700 1 --2- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b40992e0 0x7f36b41492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:01.385 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36b8e72700 1 --2- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b40973a0 0x7f36b409ca20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f36ac006b60 tx=0x7f36ac009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:01.385 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36b1ffb700 1 -- 172.21.15.154:0/478910465 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36ac00c850 con 0x7f36b40973a0 2022-11-27T17:02:01.385 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36b1ffb700 1 -- 172.21.15.154:0/478910465 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac00c9b0 con 0x7f36b40973a0 2022-11-27T17:02:01.385 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36bb0d6700 1 -- 172.21.15.154:0/478910465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c00b180 con 0x7f36b40973a0 2022-11-27T17:02:01.386 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36b1ffb700 1 -- 172.21.15.154:0/478910465 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36ac00cd10 con 0x7f36b40973a0 2022-11-27T17:02:01.386 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36bb0d6700 1 -- 172.21.15.154:0/478910465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b4139260 con 0x7f36b40973a0 2022-11-27T17:02:01.386 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.366+0000 7f36b1ffb700 1 -- 172.21.15.154:0/478910465 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 2) v1 ==== 20942+0+0 (secure 0 0 0) 0x7f36ac022070 con 0x7f36b40973a0 2022-11-27T17:02:01.386 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.367+0000 7f36b1ffb700 1 -- 172.21.15.154:0/478910465 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f36ac02a630 con 0x7f36b40973a0 2022-11-27T17:02:01.387 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.367+0000 7f36bb0d6700 1 -- 172.21.15.154:0/478910465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b409f4e0 con 0x7f36b40973a0 2022-11-27T17:02:01.387 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.369+0000 7f36b1ffb700 1 -- 172.21.15.154:0/478910465 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f36ac019960 con 0x7f36b40973a0 2022-11-27T17:02:01.436 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.432+0000 7f36bb0d6700 1 -- 172.21.15.154:0/478910465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36b40984b0 con 0x7f36b40973a0 2022-11-27T17:02:01.438 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.433+0000 7f36b1ffb700 1 -- 172.21.15.154:0/478910465 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+3913 (secure 0 0 0) 0x7f36ac029030 con 0x7f36b40973a0 2022-11-27T17:02:01.438 INFO:teuthology.misc.health.smithi154.stdout: 2022-11-27T17:02:01.439 INFO:teuthology.misc.health.smithi154.stdout:{"epoch":6,"fsid":"5adefcc8-72cd-433b-bd69-ec1eac0a6bf2","created":"2022-11-27T17:01:59.370943+0000","modified":"2022-11-27T17:02:00.584387+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"3061801a-e670-4a13-9a7d-6c526f88ef28","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6906d403-7e5e-4ac7-9739-9178ecf56f2d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5bcff406-28bc-465d-8c76-56626ab7936a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d6f7e6fc-d825-4e61-b433-6d5cf4075863","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T17:02:01.440 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.433+0000 7f36bb0d6700 1 -- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b40973a0 msgr2=0x7f36b409ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:01.440 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.433+0000 7f36bb0d6700 1 --2- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b40973a0 0x7f36b409ca20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f36ac006b60 tx=0x7f36ac009290).stop 2022-11-27T17:02:01.440 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.433+0000 7f36bb0d6700 1 -- 172.21.15.154:0/478910465 shutdown_connections 2022-11-27T17:02:01.441 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.433+0000 7f36bb0d6700 1 --2- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f36b40992e0 0x7f36b41492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:01.441 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.433+0000 7f36bb0d6700 1 --2- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36b40973a0 0x7f36b409ca20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:01.441 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.433+0000 7f36bb0d6700 1 --2- 172.21.15.154:0/478910465 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f36b4097780 0x7f36b4148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:01.441 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.433+0000 7f36bb0d6700 1 -- 172.21.15.154:0/478910465 >> 172.21.15.154:0/478910465 conn(0x7f36b405c4f0 msgr2=0x7f36b41368e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:01.442 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.433+0000 7f36bb0d6700 1 -- 172.21.15.154:0/478910465 shutdown_connections 2022-11-27T17:02:01.442 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:01.433+0000 7f36bb0d6700 1 -- 172.21.15.154:0/478910465 wait complete. 2022-11-27T17:02:01.445 DEBUG:teuthology.misc:0 of 4 OSDs are up 2022-11-27T17:02:02.331 INFO:tasks.ceph.osd.1.smithi154.stderr:2022-11-27T17:02:02.328+0000 7f61a51a2f00 -1 osd.1 0 log_to_monitors {default=true} 2022-11-27T17:02:02.337 INFO:tasks.ceph.osd.2.smithi154.stderr:2022-11-27T17:02:02.335+0000 7f1308056f00 -1 osd.2 0 log_to_monitors {default=true} 2022-11-27T17:02:02.347 INFO:tasks.ceph.osd.0.smithi154.stderr:2022-11-27T17:02:02.344+0000 7f28a8f57f00 -1 osd.0 0 log_to_monitors {default=true} 2022-11-27T17:02:02.348 INFO:tasks.ceph.osd.3.smithi154.stderr:2022-11-27T17:02:02.344+0000 7f15b93c0f00 -1 osd.3 0 log_to_monitors {default=true} 2022-11-27T17:02:04.306 INFO:tasks.ceph.osd.2.smithi154.stderr:2022-11-27T17:02:04.302+0000 7f12f0527700 -1 osd.2 0 waiting for initial osdmap 2022-11-27T17:02:04.307 INFO:tasks.ceph.osd.1.smithi154.stderr:2022-11-27T17:02:04.303+0000 7f618d673700 -1 osd.1 0 waiting for initial osdmap 2022-11-27T17:02:04.308 INFO:tasks.ceph.osd.3.smithi154.stderr:2022-11-27T17:02:04.304+0000 7f15a1891700 -1 osd.3 0 waiting for initial osdmap 2022-11-27T17:02:04.309 INFO:tasks.ceph.osd.0.smithi154.stderr:2022-11-27T17:02:04.304+0000 7f2891428700 -1 osd.0 0 waiting for initial osdmap 2022-11-27T17:02:04.314 INFO:tasks.ceph.osd.0.smithi154.stderr:2022-11-27T17:02:04.310+0000 7f289ce45700 -1 osd.0 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T17:02:04.315 INFO:tasks.ceph.osd.2.smithi154.stderr:2022-11-27T17:02:04.310+0000 7f12fbf44700 -1 osd.2 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T17:02:04.316 INFO:tasks.ceph.osd.3.smithi154.stderr:2022-11-27T17:02:04.310+0000 7f15ad2ae700 -1 osd.3 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T17:02:04.316 INFO:tasks.ceph.osd.1.smithi154.stderr:2022-11-27T17:02:04.310+0000 7f6199090700 -1 osd.1 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T17:02:07.851 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T17:02:07.957 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.954+0000 7f23499ac700 1 Processor -- start 2022-11-27T17:02:07.957 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.954+0000 7f23499ac700 1 -- start start 2022-11-27T17:02:07.964 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23499ac700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344139710 0x7f23440aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:07.965 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23499ac700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2344139af0 0x7f2344139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:07.965 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23499ac700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344135ed0 0x7f2344136380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:07.966 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23499ac700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23440aada0 con 0x7f2344139af0 2022-11-27T17:02:07.966 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23499ac700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234405d040 con 0x7f2344135ed0 2022-11-27T17:02:07.967 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23499ac700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2344139440 con 0x7f2344139710 2022-11-27T17:02:07.967 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f2342ffd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344139710 0x7f23440aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:07.968 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23437fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344135ed0 0x7f2344136380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:07.968 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f2342ffd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344139710 0x7f23440aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55000/0 (socket says 172.21.15.154:55000) 2022-11-27T17:02:07.968 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23427fc700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2344139af0 0x7f2344139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:07.968 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23437fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344135ed0 0x7f2344136380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:60948/0 (socket says 172.21.15.154:60948) 2022-11-27T17:02:07.969 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f2342ffd700 1 -- 172.21.15.154:0/3926514118 learned_addr learned my addr 172.21.15.154:0/3926514118 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:07.969 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23437fe700 1 -- 172.21.15.154:0/3926514118 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344139710 msgr2=0x7f23440aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:07.969 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23437fe700 1 --2- 172.21.15.154:0/3926514118 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344139710 0x7f23440aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:07.969 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23437fe700 1 -- 172.21.15.154:0/3926514118 >> v2:172.21.15.154:3300/0 conn(0x7f2344139af0 msgr2=0x7f2344139ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:07.970 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23437fe700 1 --2- 172.21.15.154:0/3926514118 >> v2:172.21.15.154:3300/0 conn(0x7f2344139af0 0x7f2344139ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:07.970 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23437fe700 1 -- 172.21.15.154:0/3926514118 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440aafa0 con 0x7f2344135ed0 2022-11-27T17:02:07.970 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.955+0000 7f23427fc700 1 --2- 172.21.15.154:0/3926514118 >> v2:172.21.15.154:3300/0 conn(0x7f2344139af0 0x7f2344139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:07.970 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f23437fe700 1 --2- 172.21.15.154:0/3926514118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344135ed0 0x7f2344136380 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f233c0032f0 tx=0x7f233c0091d0).ready entity=mon.1 client_cookie=dc03a0fbf470ef03 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:07.971 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f2341ffb700 1 -- 172.21.15.154:0/3926514118 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f233c00b600 con 0x7f2344135ed0 2022-11-27T17:02:07.971 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f2341ffb700 1 -- 172.21.15.154:0/3926514118 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c00b760 con 0x7f2344135ed0 2022-11-27T17:02:07.971 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f2341ffb700 1 -- 172.21.15.154:0/3926514118 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f233c00b8d0 con 0x7f2344135ed0 2022-11-27T17:02:07.971 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f23499ac700 1 -- 172.21.15.154:0/3926514118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344135ed0 msgr2=0x7f2344136380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:07.972 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f23499ac700 1 --2- 172.21.15.154:0/3926514118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344135ed0 0x7f2344136380 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f233c0032f0 tx=0x7f233c0091d0).stop 2022-11-27T17:02:07.972 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f23499ac700 1 -- 172.21.15.154:0/3926514118 shutdown_connections 2022-11-27T17:02:07.972 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f23499ac700 1 --2- 172.21.15.154:0/3926514118 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344135ed0 0x7f2344136380 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:07.972 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f23499ac700 1 --2- 172.21.15.154:0/3926514118 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344139710 0x7f23440aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:07.973 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f23499ac700 1 --2- 172.21.15.154:0/3926514118 >> v2:172.21.15.154:3300/0 conn(0x7f2344139af0 0x7f2344139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:07.973 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f23499ac700 1 -- 172.21.15.154:0/3926514118 >> 172.21.15.154:0/3926514118 conn(0x7f234405ba80 msgr2=0x7f234405be90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:07.974 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f23499ac700 1 -- 172.21.15.154:0/3926514118 shutdown_connections 2022-11-27T17:02:07.974 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.956+0000 7f23499ac700 1 -- 172.21.15.154:0/3926514118 wait complete. 2022-11-27T17:02:07.974 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.957+0000 7f23499ac700 1 Processor -- start 2022-11-27T17:02:07.975 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.957+0000 7f23499ac700 1 -- start start 2022-11-27T17:02:07.975 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f23499ac700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344135ed0 0x7f234413c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:07.975 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f23499ac700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2344139710 0x7f2344094880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:07.976 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f23499ac700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344139af0 0x7f2344148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:07.976 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f23499ac700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234413d920 con 0x7f2344139710 2022-11-27T17:02:07.977 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f23499ac700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2344148a30 con 0x7f2344139af0 2022-11-27T17:02:07.977 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f23499ac700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f234413ce70 con 0x7f2344135ed0 2022-11-27T17:02:07.978 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f2342ffd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344135ed0 0x7f234413c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:07.978 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f23427fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2344139710 0x7f2344094880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:07.979 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f2342ffd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344135ed0 0x7f234413c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55006/0 (socket says 172.21.15.154:55006) 2022-11-27T17:02:07.979 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f2342ffd700 1 -- 172.21.15.154:0/2030444635 learned_addr learned my addr 172.21.15.154:0/2030444635 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:07.979 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f23427fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2344139710 0x7f2344094880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54048/0 (socket says 172.21.15.154:54048) 2022-11-27T17:02:07.980 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f23437fe700 1 --2- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344139af0 0x7f2344148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:07.980 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f2342ffd700 1 -- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344139af0 msgr2=0x7f2344148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:07.981 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f2342ffd700 1 --2- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344139af0 0x7f2344148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:07.981 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f2342ffd700 1 -- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2344139710 msgr2=0x7f2344094880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:07.982 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f2342ffd700 1 --2- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2344139710 0x7f2344094880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:07.982 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f2342ffd700 1 -- 172.21.15.154:0/2030444635 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233400b180 con 0x7f2344135ed0 2022-11-27T17:02:07.983 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f23427fc700 1 --2- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2344139710 0x7f2344094880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:07.983 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.958+0000 7f2342ffd700 1 --2- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344135ed0 0x7f234413c980 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f234413da60 tx=0x7f2338008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:07.984 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.959+0000 7f2333fff700 1 -- 172.21.15.154:0/2030444635 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2338013610 con 0x7f2344135ed0 2022-11-27T17:02:07.984 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.959+0000 7f2333fff700 1 -- 172.21.15.154:0/2030444635 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2338013770 con 0x7f2344135ed0 2022-11-27T17:02:07.985 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.959+0000 7f23499ac700 1 -- 172.21.15.154:0/2030444635 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233c009470 con 0x7f2344135ed0 2022-11-27T17:02:07.985 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.959+0000 7f2333fff700 1 -- 172.21.15.154:0/2030444635 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2338013ad0 con 0x7f2344135ed0 2022-11-27T17:02:07.986 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.959+0000 7f23499ac700 1 -- 172.21.15.154:0/2030444635 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23440943d0 con 0x7f2344135ed0 2022-11-27T17:02:07.986 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.959+0000 7f2333fff700 1 -- 172.21.15.154:0/2030444635 <== mon.2 v2:172.21.15.154:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f2338023070 con 0x7f2344135ed0 2022-11-27T17:02:07.986 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.960+0000 7f23499ac700 1 -- 172.21.15.154:0/2030444635 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23440946d0 con 0x7f2344135ed0 2022-11-27T17:02:07.987 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.960+0000 7f2333fff700 1 -- 172.21.15.154:0/2030444635 <== mon.2 v2:172.21.15.154:3302/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2469+0+0 (secure 0 0 0) 0x7f233804a540 con 0x7f2344135ed0 2022-11-27T17:02:07.988 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:07.963+0000 7f2333fff700 1 -- 172.21.15.154:0/2030444635 <== mon.2 v2:172.21.15.154:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f2338033320 con 0x7f2344135ed0 2022-11-27T17:02:08.076 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.073+0000 7f23499ac700 1 -- 172.21.15.154:0/2030444635 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f234406b1e0 con 0x7f2344135ed0 2022-11-27T17:02:08.077 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.074+0000 7f2333fff700 1 -- 172.21.15.154:0/2030444635 <== mon.2 v2:172.21.15.154:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+6306 (secure 0 0 0) 0x7f2338031e00 con 0x7f2344135ed0 2022-11-27T17:02:08.077 INFO:teuthology.misc.health.smithi154.stdout: 2022-11-27T17:02:08.078 INFO:teuthology.misc.health.smithi154.stdout:{"epoch":12,"fsid":"5adefcc8-72cd-433b-bd69-ec1eac0a6bf2","created":"2022-11-27T17:01:59.370943+0000","modified":"2022-11-27T17:02:07.325133+0000","last_up_change":"2022-11-27T17:02:05.307461+0000","last_in_change":"2022-11-27T17:02:05.307461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-11-27T17:02:01.324848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"3061801a-e670-4a13-9a7d-6c526f88ef28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6808","nonce":105154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6809","nonce":105154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6811","nonce":105154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6810","nonce":105154}]},"public_addr":"172.21.15.154:6808/105154","cluster_addr":"172.21.15.154:6809/105154","heartbeat_back_addr":"172.21.15.154:6811/105154","heartbeat_front_addr":"172.21.15.154:6810/105154","state":["exists","up"]},{"osd":1,"uuid":"6906d403-7e5e-4ac7-9739-9178ecf56f2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6800","nonce":105155}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6801","nonce":105155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6803","nonce":105155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6802","nonce":105155}]},"public_addr":"172.21.15.154:6800/105155","cluster_addr":"172.21.15.154:6801/105155","heartbeat_back_addr":"172.21.15.154:6803/105155","heartbeat_front_addr":"172.21.15.154:6802/105155","state":["exists","up"]},{"osd":2,"uuid":"5bcff406-28bc-465d-8c76-56626ab7936a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6804","nonce":105156}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6805","nonce":105156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6807","nonce":105156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6806","nonce":105156}]},"public_addr":"172.21.15.154:6804/105156","cluster_addr":"172.21.15.154:6805/105156","heartbeat_back_addr":"172.21.15.154:6807/105156","heartbeat_front_addr":"172.21.15.154:6806/105156","state":["exists","up"]},{"osd":3,"uuid":"d6f7e6fc-d825-4e61-b433-6d5cf4075863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6812","nonce":105157}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6813","nonce":105157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6815","nonce":105157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6814","nonce":105157}]},"public_addr":"172.21.15.154:6812/105157","cluster_addr":"172.21.15.154:6813/105157","heartbeat_back_addr":"172.21.15.154:6815/105157","heartbeat_front_addr":"172.21.15.154:6814/105157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.393188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.290316+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.346843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.384664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T17:02:08.079 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.075+0000 7f23499ac700 1 -- 172.21.15.154:0/2030444635 >> v1:172.21.15.154:6816/104574 conn(0x7f2320038da0 legacy=0x7f232003b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.080 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.075+0000 7f23499ac700 1 -- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344135ed0 msgr2=0x7f234413c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.080 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.075+0000 7f23499ac700 1 --2- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344135ed0 0x7f234413c980 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f234413da60 tx=0x7f2338008040).stop 2022-11-27T17:02:08.080 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.075+0000 7f23499ac700 1 -- 172.21.15.154:0/2030444635 shutdown_connections 2022-11-27T17:02:08.080 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.075+0000 7f23499ac700 1 --2- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f2344139af0 0x7f2344148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.081 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.075+0000 7f23499ac700 1 --2- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f2344135ed0 0x7f234413c980 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.081 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.075+0000 7f23499ac700 1 --2- 172.21.15.154:0/2030444635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2344139710 0x7f2344094880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.082 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.075+0000 7f23499ac700 1 -- 172.21.15.154:0/2030444635 >> 172.21.15.154:0/2030444635 conn(0x7f234405ba80 msgr2=0x7f234409bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:08.082 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.075+0000 7f23499ac700 1 -- 172.21.15.154:0/2030444635 shutdown_connections 2022-11-27T17:02:08.082 INFO:teuthology.misc.health.smithi154.stderr:2022-11-27T17:02:08.075+0000 7f23499ac700 1 -- 172.21.15.154:0/2030444635 wait complete. 2022-11-27T17:02:08.088 DEBUG:teuthology.misc:4 of 4 OSDs are up 2022-11-27T17:02:08.088 INFO:tasks.ceph:Creating RBD pool 2022-11-27T17:02:08.089 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd pool create rbd 8 2022-11-27T17:02:08.191 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.188+0000 7fae34382700 1 Processor -- start 2022-11-27T17:02:08.191 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.188+0000 7fae34382700 1 -- start start 2022-11-27T17:02:08.198 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.188+0000 7fae34382700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fae2c1379f0 0x7fae2c137dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.199 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.188+0000 7fae34382700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c065430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.200 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.188+0000 7fae34382700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c063da0 0x7fae2c05ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.200 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.188+0000 7fae34382700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2c1382c0 con 0x7fae2c1379f0 2022-11-27T17:02:08.200 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.188+0000 7fae34382700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c137510 con 0x7fae2c063da0 2022-11-27T17:02:08.200 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.188+0000 7fae34382700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c1377d0 con 0x7fae2c063980 2022-11-27T17:02:08.201 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3291f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c063da0 0x7fae2c05ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.202 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3191d700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c065430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.202 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3291f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c063da0 0x7fae2c05ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:60974/0 (socket says 172.21.15.154:60974) 2022-11-27T17:02:08.203 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3191d700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c065430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55018/0 (socket says 172.21.15.154:55018) 2022-11-27T17:02:08.203 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3211e700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fae2c1379f0 0x7fae2c137dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.204 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3291f700 1 -- 172.21.15.154:0/2519778483 learned_addr learned my addr 172.21.15.154:0/2519778483 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:08.205 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3291f700 1 -- 172.21.15.154:0/2519778483 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 msgr2=0x7fae2c065430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:08.205 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3291f700 1 --2- 172.21.15.154:0/2519778483 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c065430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.205 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3291f700 1 -- 172.21.15.154:0/2519778483 >> v2:172.21.15.154:3300/0 conn(0x7fae2c1379f0 msgr2=0x7fae2c137dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:08.206 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3291f700 1 --2- 172.21.15.154:0/2519778483 >> v2:172.21.15.154:3300/0 conn(0x7fae2c1379f0 0x7fae2c137dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.206 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3291f700 1 -- 172.21.15.154:0/2519778483 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c064c50 con 0x7fae2c063da0 2022-11-27T17:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3211e700 1 --2- 172.21.15.154:0/2519778483 >> v2:172.21.15.154:3300/0 conn(0x7fae2c1379f0 0x7fae2c137dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:08.207 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3291f700 1 --2- 172.21.15.154:0/2519778483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c063da0 0x7fae2c05ba50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fae240032f0 tx=0x7fae24009290).ready entity=mon.1 client_cookie=e7ca716b98928b78 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:08.208 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3111c700 1 -- 172.21.15.154:0/2519778483 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae2400b6c0 con 0x7fae2c063da0 2022-11-27T17:02:08.208 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3111c700 1 -- 172.21.15.154:0/2519778483 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2400b820 con 0x7fae2c063da0 2022-11-27T17:02:08.209 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.189+0000 7fae3111c700 1 -- 172.21.15.154:0/2519778483 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae2400b990 con 0x7fae2c063da0 2022-11-27T17:02:08.209 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.190+0000 7fae34382700 1 -- 172.21.15.154:0/2519778483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c063da0 msgr2=0x7fae2c05ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.210 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.190+0000 7fae34382700 1 --2- 172.21.15.154:0/2519778483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c063da0 0x7fae2c05ba50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fae240032f0 tx=0x7fae24009290).stop 2022-11-27T17:02:08.210 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.190+0000 7fae34382700 1 -- 172.21.15.154:0/2519778483 shutdown_connections 2022-11-27T17:02:08.210 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.190+0000 7fae34382700 1 --2- 172.21.15.154:0/2519778483 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c063da0 0x7fae2c05ba50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.211 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.190+0000 7fae34382700 1 --2- 172.21.15.154:0/2519778483 >> v2:172.21.15.154:3300/0 conn(0x7fae2c1379f0 0x7fae2c137dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.211 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.190+0000 7fae34382700 1 --2- 172.21.15.154:0/2519778483 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c065430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.212 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.190+0000 7fae34382700 1 -- 172.21.15.154:0/2519778483 >> 172.21.15.154:0/2519778483 conn(0x7fae2c0616e0 msgr2=0x7fae2c05cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:08.212 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.190+0000 7fae34382700 1 -- 172.21.15.154:0/2519778483 shutdown_connections 2022-11-27T17:02:08.213 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.190+0000 7fae34382700 1 -- 172.21.15.154:0/2519778483 wait complete. 2022-11-27T17:02:08.213 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae34382700 1 Processor -- start 2022-11-27T17:02:08.214 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae34382700 1 -- start start 2022-11-27T17:02:08.214 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae34382700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c095900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.214 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae34382700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c1379f0 0x7fae2c144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.215 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae34382700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae2c144fe0 0x7fae2c145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.215 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae34382700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c067f80 con 0x7fae2c144fe0 2022-11-27T17:02:08.216 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae34382700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c095df0 con 0x7fae2c1379f0 2022-11-27T17:02:08.216 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae34382700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c145960 con 0x7fae2c063980 2022-11-27T17:02:08.217 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae3211e700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c095900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.217 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae3211e700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c095900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55020/0 (socket says 172.21.15.154:55020) 2022-11-27T17:02:08.218 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae3211e700 1 -- 172.21.15.154:0/4190328656 learned_addr learned my addr 172.21.15.154:0/4190328656 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:08.218 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae3291f700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae2c144fe0 0x7fae2c145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.218 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.191+0000 7fae3191d700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c1379f0 0x7fae2c144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.219 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae3191d700 1 -- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 msgr2=0x7fae2c095900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.220 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae3191d700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c095900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.220 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae3191d700 1 -- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae2c144fe0 msgr2=0x7fae2c145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.220 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae3191d700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae2c144fe0 0x7fae2c145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.221 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae3191d700 1 -- 172.21.15.154:0/4190328656 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2800b180 con 0x7fae2c1379f0 2022-11-27T17:02:08.221 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae3211e700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c095900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:08.222 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae3291f700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae2c144fe0 0x7fae2c145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:08.222 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae3191d700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c1379f0 0x7fae2c144af0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fae2c0955f0 tx=0x7fae1c00f040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:08.223 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae1b7fe700 1 -- 172.21.15.154:0/4190328656 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae1c018650 con 0x7fae2c1379f0 2022-11-27T17:02:08.223 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae1b7fe700 1 -- 172.21.15.154:0/4190328656 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c013040 con 0x7fae2c1379f0 2022-11-27T17:02:08.224 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae1b7fe700 1 -- 172.21.15.154:0/4190328656 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae1c0189a0 con 0x7fae2c1379f0 2022-11-27T17:02:08.224 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae34382700 1 -- 172.21.15.154:0/4190328656 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae24009530 con 0x7fae2c1379f0 2022-11-27T17:02:08.224 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.192+0000 7fae34382700 1 -- 172.21.15.154:0/4190328656 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c093920 con 0x7fae2c1379f0 2022-11-27T17:02:08.225 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.193+0000 7fae1b7fe700 1 -- 172.21.15.154:0/4190328656 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fae1c01f030 con 0x7fae2c1379f0 2022-11-27T17:02:08.225 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.193+0000 7fae1b7fe700 1 -- 172.21.15.154:0/4190328656 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2469+0+0 (secure 0 0 0) 0x7fae1c04e0e0 con 0x7fae2c1379f0 2022-11-27T17:02:08.226 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.193+0000 7fae34382700 1 -- 172.21.15.154:0/4190328656 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c0646b0 con 0x7fae2c1379f0 2022-11-27T17:02:08.228 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.197+0000 7fae1b7fe700 1 -- 172.21.15.154:0/4190328656 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fae1c036ec0 con 0x7fae2c1379f0 2022-11-27T17:02:08.310 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.307+0000 7fae34382700 1 -- 172.21.15.154:0/4190328656 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fae2c057a60 con 0x7fae2c1379f0 2022-11-27T17:02:08.448 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.444+0000 7fae1b7fe700 1 -- 172.21.15.154:0/4190328656 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (secure 0 0 0) 0x7fae1c02a070 con 0x7fae2c1379f0 2022-11-27T17:02:08.452 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.448+0000 7fae34382700 1 -- 172.21.15.154:0/4190328656 >> v1:172.21.15.154:6816/104574 conn(0x7fae0c038da0 legacy=0x7fae0c03b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.452 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.448+0000 7fae34382700 1 -- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c1379f0 msgr2=0x7fae2c144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.453 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.448+0000 7fae34382700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c1379f0 0x7fae2c144af0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fae2c0955f0 tx=0x7fae1c00f040).stop 2022-11-27T17:02:08.454 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.448+0000 7fae34382700 1 -- 172.21.15.154:0/4190328656 shutdown_connections 2022-11-27T17:02:08.454 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.448+0000 7fae34382700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae2c144fe0 0x7fae2c145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.454 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.448+0000 7fae34382700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fae2c063980 0x7fae2c095900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.455 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.448+0000 7fae34382700 1 --2- 172.21.15.154:0/4190328656 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fae2c1379f0 0x7fae2c144af0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.455 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.448+0000 7fae34382700 1 -- 172.21.15.154:0/4190328656 >> 172.21.15.154:0/4190328656 conn(0x7fae2c0616e0 msgr2=0x7fae2c0a91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:08.456 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.448+0000 7fae34382700 1 -- 172.21.15.154:0/4190328656 shutdown_connections 2022-11-27T17:02:08.456 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.449+0000 7fae34382700 1 -- 172.21.15.154:0/4190328656 wait complete. 2022-11-27T17:02:08.456 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'rbd' created 2022-11-27T17:02:08.462 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-11-27T17:02:08.577 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.565+0000 7f56ab455700 1 Processor -- start 2022-11-27T17:02:08.577 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.565+0000 7f56ab455700 1 -- start start 2022-11-27T17:02:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56ab455700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56a4098440 0x7f56a4136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56ab455700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 0x7f56a4140b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56ab455700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a4099050 con 0x7f56a41366d0 2022-11-27T17:02:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56ab455700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a4097cb0 con 0x7f56a4098440 2022-11-27T17:02:08.579 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56ab455700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a4136fa0 con 0x7f56a413e780 2022-11-27T17:02:08.579 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a99f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 0x7f56a4140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a89f0700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f56a4098440 0x7f56a4136ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a99f2700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 0x7f56a4140b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49248/0 (socket says 172.21.15.154:49248) 2022-11-27T17:02:08.621 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a99f2700 1 -- 172.21.15.154:0/1223185936 learned_addr learned my addr 172.21.15.154:0/1223185936 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:08.621 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a91f1700 1 --1- 172.21.15.154:0/1223185936 >> v1:172.21.15.154:6789/0 conn(0x7f56a41366d0 0x7f56a40975a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40536/0 (socket says 172.21.15.154:40536) 2022-11-27T17:02:08.622 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a89f0700 1 -- 172.21.15.154:0/1223185936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 msgr2=0x7f56a4140b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:08.622 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a89f0700 1 --2- 172.21.15.154:0/1223185936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 0x7f56a4140b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.622 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a89f0700 1 -- 172.21.15.154:0/1223185936 >> v1:172.21.15.154:6789/0 conn(0x7f56a41366d0 legacy=0x7f56a40975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.623 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f5693fff700 1 -- 172.21.15.154:0/1223185936 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599992456 0 0) 0x7f56a4099050 con 0x7f56a41366d0 2022-11-27T17:02:08.624 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a89f0700 1 -- 172.21.15.154:0/1223185936 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a4099920 con 0x7f56a4098440 2022-11-27T17:02:08.624 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a99f2700 1 --2- 172.21.15.154:0/1223185936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 0x7f56a4140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:08.625 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f56a89f0700 1 --2- 172.21.15.154:0/1223185936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 0x7f56a4136ab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5694002f70 tx=0x7f5694006df0).ready entity=mon.0 client_cookie=7f45de3f8aeec94c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:08.625 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f5693fff700 1 -- 172.21.15.154:0/1223185936 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f569400d570 con 0x7f56a4098440 2022-11-27T17:02:08.625 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f5693fff700 1 -- 172.21.15.154:0/1223185936 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5694000e90 con 0x7f56a4098440 2022-11-27T17:02:08.626 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.566+0000 7f5693fff700 1 -- 172.21.15.154:0/1223185936 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f569400d8c0 con 0x7f56a4098440 2022-11-27T17:02:08.626 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.567+0000 7f56ab455700 1 -- 172.21.15.154:0/1223185936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 msgr2=0x7f56a4136ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.627 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.567+0000 7f56ab455700 1 --2- 172.21.15.154:0/1223185936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 0x7f56a4136ab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f5694002f70 tx=0x7f5694006df0).stop 2022-11-27T17:02:08.627 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.567+0000 7f56ab455700 1 -- 172.21.15.154:0/1223185936 shutdown_connections 2022-11-27T17:02:08.628 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.567+0000 7f56ab455700 1 --2- 172.21.15.154:0/1223185936 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 0x7f56a4140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.628 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.567+0000 7f56ab455700 1 --2- 172.21.15.154:0/1223185936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 0x7f56a4136ab0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.629 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.567+0000 7f56ab455700 1 -- 172.21.15.154:0/1223185936 >> 172.21.15.154:0/1223185936 conn(0x7f56a4067f70 msgr2=0x7f56a4060d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:08.629 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.567+0000 7f56ab455700 1 -- 172.21.15.154:0/1223185936 shutdown_connections 2022-11-27T17:02:08.629 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.567+0000 7f56ab455700 1 -- 172.21.15.154:0/1223185936 wait complete. 2022-11-27T17:02:08.630 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56ab455700 1 Processor -- start 2022-11-27T17:02:08.630 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56ab455700 1 -- start start 2022-11-27T17:02:08.631 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56ab455700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 0x7f56a4065b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.631 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56ab455700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f56a41366d0 0x7f56a4062a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.632 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56ab455700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 0x7f56a4062f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:08.632 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56ab455700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a409afe0 con 0x7f56a4098440 2022-11-27T17:02:08.633 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56ab455700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a40661c0 con 0x7f56a413e780 2022-11-27T17:02:08.633 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56ab455700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56a4066480 con 0x7f56a41366d0 2022-11-27T17:02:08.634 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56a91f1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 0x7f56a4065b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.634 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56a89f0700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f56a41366d0 0x7f56a4062a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.634 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56a91f1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 0x7f56a4065b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49916/0 (socket says 172.21.15.154:49916) 2022-11-27T17:02:08.634 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56a91f1700 1 -- 172.21.15.154:0/4217775462 learned_addr learned my addr 172.21.15.154:0/4217775462 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:08.635 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.568+0000 7f56a99f2700 1 --2- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 0x7f56a4062f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:08.635 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56a91f1700 1 -- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f56a41366d0 msgr2=0x7f56a4062a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.635 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56a91f1700 1 --2- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f56a41366d0 0x7f56a4062a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.635 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56a91f1700 1 -- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 msgr2=0x7f56a4062f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:08.636 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56a91f1700 1 --2- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 0x7f56a4062f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:08.636 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56a91f1700 1 -- 172.21.15.154:0/4217775462 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c009530 con 0x7f56a4098440 2022-11-27T17:02:08.636 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56a91f1700 1 --2- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 0x7f56a4065b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f56a0000bc0 tx=0x7f56a0007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:08.637 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56927fc700 1 -- 172.21.15.154:0/4217775462 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56a00070d0 con 0x7f56a4098440 2022-11-27T17:02:08.637 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56927fc700 1 -- 172.21.15.154:0/4217775462 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a0007230 con 0x7f56a4098440 2022-11-27T17:02:08.637 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56ab455700 1 -- 172.21.15.154:0/4217775462 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569400b180 con 0x7f56a4098440 2022-11-27T17:02:08.638 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56927fc700 1 -- 172.21.15.154:0/4217775462 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56a00056d0 con 0x7f56a4098440 2022-11-27T17:02:08.638 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.569+0000 7f56ab455700 1 -- 172.21.15.154:0/4217775462 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a4064160 con 0x7f56a4098440 2022-11-27T17:02:08.638 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.570+0000 7f56927fc700 1 -- 172.21.15.154:0/4217775462 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f56a001a060 con 0x7f56a4098440 2022-11-27T17:02:08.639 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.570+0000 7f56ab455700 1 -- 172.21.15.154:0/4217775462 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a4099be0 con 0x7f56a4098440 2022-11-27T17:02:08.639 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.570+0000 7f56927fc700 1 -- 172.21.15.154:0/4217775462 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2811+0+0 (secure 0 0 0) 0x7f56a00497d0 con 0x7f56a4098440 2022-11-27T17:02:08.639 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.574+0000 7f56927fc700 1 -- 172.21.15.154:0/4217775462 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f56a0018080 con 0x7f56a4098440 2022-11-27T17:02:08.688 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:08.685+0000 7f56ab455700 1 -- 172.21.15.154:0/4217775462 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f56a4057a60 con 0x7f56a4098440 2022-11-27T17:02:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.450+0000 7f56927fc700 1 -- 172.21.15.154:0/4217775462 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 173+0+0 (secure 0 0 0) 0x7f56a002ea30 con 0x7f56a4098440 2022-11-27T17:02:09.457 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.453+0000 7f56ab455700 1 -- 172.21.15.154:0/4217775462 >> v1:172.21.15.154:6816/104574 conn(0x7f5684038da0 legacy=0x7f568403b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.457 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.453+0000 7f56ab455700 1 -- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 msgr2=0x7f56a4065b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.457 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.453+0000 7f56ab455700 1 --2- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 0x7f56a4065b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f56a0000bc0 tx=0x7f56a0007bc0).stop 2022-11-27T17:02:09.457 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.453+0000 7f56ab455700 1 -- 172.21.15.154:0/4217775462 shutdown_connections 2022-11-27T17:02:09.458 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.453+0000 7f56ab455700 1 --2- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f56a413e780 0x7f56a4062f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.458 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.453+0000 7f56ab455700 1 --2- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56a4098440 0x7f56a4065b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.458 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.453+0000 7f56ab455700 1 --2- 172.21.15.154:0/4217775462 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f56a41366d0 0x7f56a4062a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.458 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.454+0000 7f56ab455700 1 -- 172.21.15.154:0/4217775462 >> 172.21.15.154:0/4217775462 conn(0x7f56a4067f70 msgr2=0x7f56a40a8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:09.458 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.454+0000 7f56ab455700 1 -- 172.21.15.154:0/4217775462 shutdown_connections 2022-11-27T17:02:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.454+0000 7f56ab455700 1 -- 172.21.15.154:0/4217775462 wait complete. 2022-11-27T17:02:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:enabled application 'rbd' on pool 'rbd' 2022-11-27T17:02:09.468 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-11-27T17:02:09.469 INFO:tasks.ceph.mds.a:Restarting daemon 2022-11-27T17:02:09.469 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-11-27T17:02:09.472 INFO:tasks.ceph.mds.a:Started 2022-11-27T17:02:09.472 INFO:tasks.ceph.mds.b:Restarting daemon 2022-11-27T17:02:09.472 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-11-27T17:02:09.474 INFO:tasks.ceph.mds.b:Started 2022-11-27T17:02:09.475 INFO:tasks.ceph.mds.c:Restarting daemon 2022-11-27T17:02:09.475 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-11-27T17:02:09.477 INFO:tasks.ceph.mds.c:Started 2022-11-27T17:02:09.478 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-11-27T17:02:09.478 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T17:02:09.554 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2022-11-27T17:02:09.562 INFO:tasks.ceph.mds.b.smithi154.stdout:starting mds.b at 2022-11-27T17:02:09.563 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2022-11-27T17:02:09.607 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.593+0000 7f7239543700 1 Processor -- start 2022-11-27T17:02:09.608 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.593+0000 7f7239543700 1 -- start start 2022-11-27T17:02:09.608 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f7239543700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 0x7f723409abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.608 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f7239543700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234098100 0x7f7234140640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.609 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f7239543700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723409a080 con 0x7f7234097d00 2022-11-27T17:02:09.609 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f7239543700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723409b0c0 con 0x7f7234098100 2022-11-27T17:02:09.610 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f7239543700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f723409a320 con 0x7f72340990a0 2022-11-27T17:02:09.610 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f72337fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234098100 0x7f7234140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.610 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f72327fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7234097d00 0x7f72341366c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40590/0 (socket says 172.21.15.154:40590) 2022-11-27T17:02:09.611 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f7232ffd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 0x7f723409abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.611 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f72327fc700 1 -- 172.21.15.154:0/1196901533 learned_addr learned my addr 172.21.15.154:0/1196901533 (peer_addr_for_me v1:172.21.15.154:0/0) 2022-11-27T17:02:09.612 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f7231ffb700 1 -- 172.21.15.154:0/1196901533 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162745948 0 0) 0x7f723409a080 con 0x7f7234097d00 2022-11-27T17:02:09.612 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f7231ffb700 1 -- 172.21.15.154:0/1196901533 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7218003260 con 0x7f7234097d00 2022-11-27T17:02:09.613 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f72337fe700 1 -- 172.21.15.154:0/1196901533 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 msgr2=0x7f723409abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:09.613 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f72337fe700 1 --2- 172.21.15.154:0/1196901533 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 0x7f723409abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.614 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f72337fe700 1 -- 172.21.15.154:0/1196901533 >> v1:172.21.15.154:6789/0 conn(0x7f7234097d00 legacy=0x7f72341366c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.614 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f72337fe700 1 -- 172.21.15.154:0/1196901533 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723405e8c0 con 0x7f7234098100 2022-11-27T17:02:09.614 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f7232ffd700 1 --2- 172.21.15.154:0/1196901533 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 0x7f723409abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:09.615 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.594+0000 7f72337fe700 1 --2- 172.21.15.154:0/1196901533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234098100 0x7f7234140640 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f722c0032f0 tx=0x7f722c0091d0).ready entity=mon.1 client_cookie=b942d75b4188de2c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:09.615 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7231ffb700 1 -- 172.21.15.154:0/1196901533 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f722c00b640 con 0x7f7234098100 2022-11-27T17:02:09.616 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7231ffb700 1 -- 172.21.15.154:0/1196901533 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722c00b7a0 con 0x7f7234098100 2022-11-27T17:02:09.616 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7231ffb700 1 -- 172.21.15.154:0/1196901533 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f722c00b910 con 0x7f7234098100 2022-11-27T17:02:09.616 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7239543700 1 -- 172.21.15.154:0/1196901533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234098100 msgr2=0x7f7234140640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.617 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7239543700 1 --2- 172.21.15.154:0/1196901533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234098100 0x7f7234140640 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f722c0032f0 tx=0x7f722c0091d0).stop 2022-11-27T17:02:09.617 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7239543700 1 -- 172.21.15.154:0/1196901533 shutdown_connections 2022-11-27T17:02:09.618 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7239543700 1 --2- 172.21.15.154:0/1196901533 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234098100 0x7f7234140640 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.618 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7239543700 1 --2- 172.21.15.154:0/1196901533 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 0x7f723409abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.618 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7239543700 1 -- 172.21.15.154:0/1196901533 >> 172.21.15.154:0/1196901533 conn(0x7f723405ba50 msgr2=0x7f723405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:09.619 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7239543700 1 -- 172.21.15.154:0/1196901533 shutdown_connections 2022-11-27T17:02:09.619 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.595+0000 7f7239543700 1 -- 172.21.15.154:0/1196901533 wait complete. 2022-11-27T17:02:09.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.596+0000 7f7239543700 1 Processor -- start 2022-11-27T17:02:09.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.596+0000 7f7239543700 1 -- start start 2022-11-27T17:02:09.621 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7239543700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234097d00 0x7f72340a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.621 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7239543700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7234098100 0x7f72340a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.621 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7239543700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 0x7f72340a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.622 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7239543700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72341461d0 con 0x7f7234098100 2022-11-27T17:02:09.622 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7239543700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723409fdf0 con 0x7f7234097d00 2022-11-27T17:02:09.623 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7239543700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72340a2750 con 0x7f72340990a0 2022-11-27T17:02:09.623 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7232ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234097d00 0x7f72340a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.624 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f72327fc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7234098100 0x7f72340a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.624 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7232ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234097d00 0x7f72340a33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49276/0 (socket says 172.21.15.154:49276) 2022-11-27T17:02:09.625 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7232ffd700 1 -- 172.21.15.154:0/850178380 learned_addr learned my addr 172.21.15.154:0/850178380 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:09.625 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f72337fe700 1 --2- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 0x7f72340a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.626 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7232ffd700 1 -- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 msgr2=0x7f72340a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.627 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7232ffd700 1 --2- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 0x7f72340a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.628 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7232ffd700 1 -- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7234098100 msgr2=0x7f72340a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.629 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7232ffd700 1 --2- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7234098100 0x7f72340a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.629 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7232ffd700 1 -- 172.21.15.154:0/850178380 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722800b180 con 0x7f7234097d00 2022-11-27T17:02:09.630 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7232ffd700 1 --2- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234097d00 0x7f72340a33c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f72340a28e0 tx=0x7f7228006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:09.630 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7223fff700 1 -- 172.21.15.154:0/850178380 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f722800c7e0 con 0x7f7234097d00 2022-11-27T17:02:09.630 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7223fff700 1 -- 172.21.15.154:0/850178380 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722800c940 con 0x7f7234097d00 2022-11-27T17:02:09.631 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7239543700 1 -- 172.21.15.154:0/850178380 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722c009470 con 0x7f7234097d00 2022-11-27T17:02:09.631 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7223fff700 1 -- 172.21.15.154:0/850178380 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f722800cca0 con 0x7f7234097d00 2022-11-27T17:02:09.632 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.597+0000 7f7239543700 1 -- 172.21.15.154:0/850178380 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72340a38b0 con 0x7f7234097d00 2022-11-27T17:02:09.632 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.598+0000 7f7223fff700 1 -- 172.21.15.154:0/850178380 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f722801c460 con 0x7f7234097d00 2022-11-27T17:02:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.598+0000 7f7239543700 1 -- 172.21.15.154:0/850178380 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72341464b0 con 0x7f7234097d00 2022-11-27T17:02:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.598+0000 7f7223fff700 1 -- 172.21.15.154:0/850178380 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f722804ab50 con 0x7f7234097d00 2022-11-27T17:02:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.602+0000 7f7223fff700 1 -- 172.21.15.154:0/850178380 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f7228030100 con 0x7f7234097d00 2022-11-27T17:02:09.715 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.711+0000 7f7239543700 1 -- 172.21.15.154:0/850178380 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7234136dd0 con 0x7f7234097d00 2022-11-27T17:02:09.716 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.712+0000 7f7223fff700 1 -- 172.21.15.154:0/850178380 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7690 (secure 0 0 0) 0x7f722800ce00 con 0x7f7234097d00 2022-11-27T17:02:09.716 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:02:09.716 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":14,"fsid":"5adefcc8-72cd-433b-bd69-ec1eac0a6bf2","created":"2022-11-27T17:01:59.370943+0000","modified":"2022-11-27T17:02:09.446137+0000","last_up_change":"2022-11-27T17:02:05.307461+0000","last_in_change":"2022-11-27T17:02:05.307461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-11-27T17:02:01.324848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T17:02:08.341335+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3061801a-e670-4a13-9a7d-6c526f88ef28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6808","nonce":105154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6809","nonce":105154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6811","nonce":105154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6810","nonce":105154}]},"public_addr":"172.21.15.154:6808/105154","cluster_addr":"172.21.15.154:6809/105154","heartbeat_back_addr":"172.21.15.154:6811/105154","heartbeat_front_addr":"172.21.15.154:6810/105154","state":["exists","up"]},{"osd":1,"uuid":"6906d403-7e5e-4ac7-9739-9178ecf56f2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6800","nonce":105155}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6801","nonce":105155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6803","nonce":105155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6802","nonce":105155}]},"public_addr":"172.21.15.154:6800/105155","cluster_addr":"172.21.15.154:6801/105155","heartbeat_back_addr":"172.21.15.154:6803/105155","heartbeat_front_addr":"172.21.15.154:6802/105155","state":["exists","up"]},{"osd":2,"uuid":"5bcff406-28bc-465d-8c76-56626ab7936a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6804","nonce":105156}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6805","nonce":105156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6807","nonce":105156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6806","nonce":105156}]},"public_addr":"172.21.15.154:6804/105156","cluster_addr":"172.21.15.154:6805/105156","heartbeat_back_addr":"172.21.15.154:6807/105156","heartbeat_front_addr":"172.21.15.154:6806/105156","state":["exists","up"]},{"osd":3,"uuid":"d6f7e6fc-d825-4e61-b433-6d5cf4075863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6812","nonce":105157}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6813","nonce":105157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6815","nonce":105157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6814","nonce":105157}]},"public_addr":"172.21.15.154:6812/105157","cluster_addr":"172.21.15.154:6813/105157","heartbeat_back_addr":"172.21.15.154:6815/105157","heartbeat_front_addr":"172.21.15.154:6814/105157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.393188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.290316+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.346843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.384664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T17:02:09.718 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.713+0000 7f7239543700 1 -- 172.21.15.154:0/850178380 >> v1:172.21.15.154:6816/104574 conn(0x7f7210038dd0 legacy=0x7f721003b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.718 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.713+0000 7f7239543700 1 -- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234097d00 msgr2=0x7f72340a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.718 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.713+0000 7f7239543700 1 --2- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234097d00 0x7f72340a33c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f72340a28e0 tx=0x7f7228006df0).stop 2022-11-27T17:02:09.719 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.713+0000 7f7239543700 1 -- 172.21.15.154:0/850178380 shutdown_connections 2022-11-27T17:02:09.719 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.713+0000 7f7239543700 1 --2- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f72340990a0 0x7f72340a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.720 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.713+0000 7f7239543700 1 --2- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f7234097d00 0x7f72340a33c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.720 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.713+0000 7f7239543700 1 --2- 172.21.15.154:0/850178380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7234098100 0x7f72340a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.720 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.713+0000 7f7239543700 1 -- 172.21.15.154:0/850178380 >> 172.21.15.154:0/850178380 conn(0x7f723405ba50 msgr2=0x7f7234061fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:09.721 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.714+0000 7f7239543700 1 -- 172.21.15.154:0/850178380 shutdown_connections 2022-11-27T17:02:09.721 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.714+0000 7f7239543700 1 -- 172.21.15.154:0/850178380 wait complete. 2022-11-27T17:02:09.729 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-11-27T17:02:01.324848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T17:02:08.341335+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T17:02:09.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-11-27T17:02:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.837+0000 7f3621770700 1 Processor -- start 2022-11-27T17:02:09.840 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.837+0000 7f3621770700 1 -- start start 2022-11-27T17:02:09.846 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f3621770700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c135d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.847 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f3621770700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.847 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f3621770700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f361c0a0ed0 0x7f361c139340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.847 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f3621770700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c0a5970 con 0x7f361c0a0ed0 2022-11-27T17:02:09.848 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f3621770700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0a6530 con 0x7f361c1333c0 2022-11-27T17:02:09.848 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f3621770700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f361c136400 con 0x7f361c060240 2022-11-27T17:02:09.848 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361affd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c135d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.849 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361a7fc700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.849 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f361c0a0ed0 0x7f361c139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.849 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361b7fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f361c0a0ed0 0x7f361c139340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49954/0 (socket says 172.21.15.154:49954) 2022-11-27T17:02:09.849 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361affd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c135d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49278/0 (socket says 172.21.15.154:49278) 2022-11-27T17:02:09.850 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361b7fe700 1 -- 172.21.15.154:0/1107723286 learned_addr learned my addr 172.21.15.154:0/1107723286 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:09.850 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361a7fc700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c0a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55164/0 (socket says 172.21.15.154:55164) 2022-11-27T17:02:09.850 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361b7fe700 1 -- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 msgr2=0x7f361c0a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:09.850 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361b7fe700 1 --2- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c0a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.851 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361b7fe700 1 -- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 msgr2=0x7f361c135d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:09.851 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361b7fe700 1 --2- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c135d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.851 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361b7fe700 1 -- 172.21.15.154:0/1107723286 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c1366a0 con 0x7f361c0a0ed0 2022-11-27T17:02:09.851 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361affd700 1 --2- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c135d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:09.852 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361a7fc700 1 --2- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c0a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:09.852 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f361b7fe700 1 --2- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c0a0ed0 0x7f361c139340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f36140032f0 tx=0x7f3614009290).ready entity=mon.0 client_cookie=5dff05f12bc3b353 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:09.852 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f3619ffb700 1 -- 172.21.15.154:0/1107723286 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361400b6c0 con 0x7f361c0a0ed0 2022-11-27T17:02:09.852 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f3619ffb700 1 -- 172.21.15.154:0/1107723286 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361400b820 con 0x7f361c0a0ed0 2022-11-27T17:02:09.853 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.838+0000 7f3619ffb700 1 -- 172.21.15.154:0/1107723286 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361400b990 con 0x7f361c0a0ed0 2022-11-27T17:02:09.853 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.839+0000 7f3621770700 1 -- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c0a0ed0 msgr2=0x7f361c139340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.853 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.839+0000 7f3621770700 1 --2- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c0a0ed0 0x7f361c139340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f36140032f0 tx=0x7f3614009290).stop 2022-11-27T17:02:09.853 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.839+0000 7f3621770700 1 -- 172.21.15.154:0/1107723286 shutdown_connections 2022-11-27T17:02:09.853 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.839+0000 7f3621770700 1 --2- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c135d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.854 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.839+0000 7f3621770700 1 --2- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c0a0ed0 0x7f361c139340 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.854 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.839+0000 7f3621770700 1 --2- 172.21.15.154:0/1107723286 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c0a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.854 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.839+0000 7f3621770700 1 -- 172.21.15.154:0/1107723286 >> 172.21.15.154:0/1107723286 conn(0x7f361c05dfb0 msgr2=0x7f361c0a07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:09.854 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.839+0000 7f3621770700 1 -- 172.21.15.154:0/1107723286 shutdown_connections 2022-11-27T17:02:09.855 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.839+0000 7f3621770700 1 -- 172.21.15.154:0/1107723286 wait complete. 2022-11-27T17:02:09.855 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.840+0000 7f3621770700 1 Processor -- start 2022-11-27T17:02:09.855 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.840+0000 7f3621770700 1 -- start start 2022-11-27T17:02:09.855 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.840+0000 7f3621770700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.856 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.840+0000 7f3621770700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c0a0ed0 0x7f361c148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.856 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f3621770700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:09.856 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f3621770700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c064490 con 0x7f361c0a0ed0 2022-11-27T17:02:09.856 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f3621770700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c139cd0 con 0x7f361c1333c0 2022-11-27T17:02:09.857 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f3621770700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f361c139f90 con 0x7f361c060240 2022-11-27T17:02:09.857 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361affd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.857 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361affd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c139610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55180/0 (socket says 172.21.15.154:55180) 2022-11-27T17:02:09.857 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361affd700 1 -- 172.21.15.154:0/1857396664 learned_addr learned my addr 172.21.15.154:0/1857396664 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:09.857 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361b7fe700 1 --2- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.858 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361a7fc700 1 --2- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c0a0ed0 0x7f361c148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:09.858 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361affd700 1 -- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 msgr2=0x7f361c1491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.858 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361affd700 1 --2- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c1491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.858 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361affd700 1 -- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c0a0ed0 msgr2=0x7f361c148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.859 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361affd700 1 --2- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c0a0ed0 0x7f361c148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.859 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361affd700 1 -- 172.21.15.154:0/1857396664 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c00b180 con 0x7f361c060240 2022-11-27T17:02:09.860 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361b7fe700 1 --2- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c1491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:09.860 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f361affd700 1 --2- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c139610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f361000c000 tx=0x7f3610006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:09.860 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f360bfff700 1 -- 172.21.15.154:0/1857396664 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3610010780 con 0x7f361c060240 2022-11-27T17:02:09.860 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f360bfff700 1 -- 172.21.15.154:0/1857396664 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36100108e0 con 0x7f361c060240 2022-11-27T17:02:09.860 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f3621770700 1 -- 172.21.15.154:0/1857396664 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361000b180 con 0x7f361c060240 2022-11-27T17:02:09.861 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f360bfff700 1 -- 172.21.15.154:0/1857396664 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3610010c40 con 0x7f361c060240 2022-11-27T17:02:09.861 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.841+0000 7f3621770700 1 -- 172.21.15.154:0/1857396664 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3614009530 con 0x7f361c060240 2022-11-27T17:02:09.861 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.842+0000 7f360bfff700 1 -- 172.21.15.154:0/1857396664 <== mon.2 v2:172.21.15.154:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f361001c460 con 0x7f361c060240 2022-11-27T17:02:09.861 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.842+0000 7f3621770700 1 -- 172.21.15.154:0/1857396664 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c1337a0 con 0x7f361c060240 2022-11-27T17:02:09.862 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.842+0000 7f360bfff700 1 -- 172.21.15.154:0/1857396664 <== mon.2 v2:172.21.15.154:3302/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f361004a9b0 con 0x7f361c060240 2022-11-27T17:02:09.862 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.846+0000 7f360bfff700 1 -- 172.21.15.154:0/1857396664 <== mon.2 v2:172.21.15.154:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f3610033630 con 0x7f361c060240 2022-11-27T17:02:09.963 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.959+0000 7f3621770700 1 -- 172.21.15.154:0/1857396664 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f361c057a60 con 0x7f361c060240 2022-11-27T17:02:09.964 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.959+0000 7f360bfff700 1 -- 172.21.15.154:0/1857396664 <== mon.2 v2:172.21.15.154:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v14) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3610033220 con 0x7f361c060240 2022-11-27T17:02:09.964 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2022-11-27T17:02:09.966 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.961+0000 7f3621770700 1 -- 172.21.15.154:0/1857396664 >> v1:172.21.15.154:6816/104574 conn(0x7f35f8038dd0 legacy=0x7f35f803b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.966 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.961+0000 7f3621770700 1 -- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 msgr2=0x7f361c139610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:09.966 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.961+0000 7f3621770700 1 --2- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c139610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f361000c000 tx=0x7f3610006df0).stop 2022-11-27T17:02:09.966 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.961+0000 7f3621770700 1 -- 172.21.15.154:0/1857396664 shutdown_connections 2022-11-27T17:02:09.967 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.961+0000 7f3621770700 1 --2- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f361c1333c0 0x7f361c1491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.967 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.961+0000 7f3621770700 1 --2- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f361c060240 0x7f361c139610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.967 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.961+0000 7f3621770700 1 --2- 172.21.15.154:0/1857396664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c0a0ed0 0x7f361c148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:09.967 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.961+0000 7f3621770700 1 -- 172.21.15.154:0/1857396664 >> 172.21.15.154:0/1857396664 conn(0x7f361c05dfb0 msgr2=0x7f361c0a7310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:09.967 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.961+0000 7f3621770700 1 -- 172.21.15.154:0/1857396664 shutdown_connections 2022-11-27T17:02:09.968 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:09.961+0000 7f3621770700 1 -- 172.21.15.154:0/1857396664 wait complete. 2022-11-27T17:02:09.975 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T17:02:10.083 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.080+0000 7f0985dea700 1 Processor -- start 2022-11-27T17:02:10.083 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.080+0000 7f0985dea700 1 -- start start 2022-11-27T17:02:10.090 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.080+0000 7f0985dea700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09801333c0 0x7f0980135d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.090 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.080+0000 7f0985dea700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0980060240 0x7f09800a5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.092 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.080+0000 7f0985dea700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09800a6530 con 0x7f09800a0ed0 2022-11-27T17:02:10.092 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f0985dea700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09800a5970 con 0x7f0980060240 2022-11-27T17:02:10.092 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f0985dea700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0980136400 con 0x7f09801333c0 2022-11-27T17:02:10.093 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09801333c0 0x7f0980135d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.093 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f0976ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0980060240 0x7f09800a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.094 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09801333c0 0x7f0980135d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55190/0 (socket says 172.21.15.154:55190) 2022-11-27T17:02:10.094 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f097f7fe700 1 -- 172.21.15.154:0/4020135659 learned_addr learned my addr 172.21.15.154:0/4020135659 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:10.094 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f097ffff700 1 --1- 172.21.15.154:0/4020135659 >> v1:172.21.15.154:6789/0 conn(0x7f09800a0ed0 0x7f0980139340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40606/0 (socket says 172.21.15.154:40606) 2022-11-27T17:02:10.094 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f097effd700 1 -- 172.21.15.154:0/4020135659 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596973832 0 0) 0x7f09800a6530 con 0x7f09800a0ed0 2022-11-27T17:02:10.095 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f0976ffd700 1 -- 172.21.15.154:0/4020135659 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09801333c0 msgr2=0x7f0980135d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:10.095 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f0976ffd700 1 --2- 172.21.15.154:0/4020135659 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09801333c0 0x7f0980135d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.095 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f0976ffd700 1 -- 172.21.15.154:0/4020135659 >> v1:172.21.15.154:6789/0 conn(0x7f09800a0ed0 legacy=0x7f0980139340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.095 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f0976ffd700 1 -- 172.21.15.154:0/4020135659 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09801366a0 con 0x7f0980060240 2022-11-27T17:02:10.096 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f0976ffd700 1 --2- 172.21.15.154:0/4020135659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0980060240 0x7f09800a5480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f0968002f70 tx=0x7f0968006df0).ready entity=mon.1 client_cookie=2dfe1a6552f64451 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:10.096 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f097effd700 1 -- 172.21.15.154:0/4020135659 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f096800d570 con 0x7f0980060240 2022-11-27T17:02:10.096 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f097effd700 1 -- 172.21.15.154:0/4020135659 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0968000e90 con 0x7f0980060240 2022-11-27T17:02:10.096 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.081+0000 7f097effd700 1 -- 172.21.15.154:0/4020135659 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f096800d8c0 con 0x7f0980060240 2022-11-27T17:02:10.097 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.082+0000 7f0985dea700 1 -- 172.21.15.154:0/4020135659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0980060240 msgr2=0x7f09800a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.097 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.082+0000 7f0985dea700 1 --2- 172.21.15.154:0/4020135659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0980060240 0x7f09800a5480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f0968002f70 tx=0x7f0968006df0).stop 2022-11-27T17:02:10.097 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.082+0000 7f0985dea700 1 -- 172.21.15.154:0/4020135659 shutdown_connections 2022-11-27T17:02:10.097 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.082+0000 7f0985dea700 1 --2- 172.21.15.154:0/4020135659 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09801333c0 0x7f0980135d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.098 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.082+0000 7f0985dea700 1 --2- 172.21.15.154:0/4020135659 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0980060240 0x7f09800a5480 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.098 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.082+0000 7f0985dea700 1 -- 172.21.15.154:0/4020135659 >> 172.21.15.154:0/4020135659 conn(0x7f098005dfb0 msgr2=0x7f09800a07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:10.098 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.082+0000 7f0985dea700 1 -- 172.21.15.154:0/4020135659 shutdown_connections 2022-11-27T17:02:10.098 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.082+0000 7f0985dea700 1 -- 172.21.15.154:0/4020135659 wait complete. 2022-11-27T17:02:10.099 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.083+0000 7f0985dea700 1 Processor -- start 2022-11-27T17:02:10.099 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.083+0000 7f0985dea700 1 -- start start 2022-11-27T17:02:10.099 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.083+0000 7f0985dea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0980060240 0x7f0980063e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.100 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.083+0000 7f0985dea700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09800a0ed0 0x7f0980137980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.100 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.083+0000 7f0985dea700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09801333c0 0x7f0980137e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.100 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.083+0000 7f0985dea700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0980139ab0 con 0x7f0980060240 2022-11-27T17:02:10.101 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.083+0000 7f0985dea700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09801376c0 con 0x7f09801333c0 2022-11-27T17:02:10.101 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.083+0000 7f0985dea700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0980139870 con 0x7f09800a0ed0 2022-11-27T17:02:10.101 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.083+0000 7f097ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09801333c0 0x7f0980137e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.102 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.083+0000 7f0976ffd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09800a0ed0 0x7f0980137980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.103 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097ffff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09801333c0 0x7f0980137e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49304/0 (socket says 172.21.15.154:49304) 2022-11-27T17:02:10.103 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097ffff700 1 -- 172.21.15.154:0/3429960831 learned_addr learned my addr 172.21.15.154:0/3429960831 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:10.103 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f0976ffd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09800a0ed0 0x7f0980137980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55206/0 (socket says 172.21.15.154:55206) 2022-11-27T17:02:10.104 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097f7fe700 1 --2- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0980060240 0x7f0980063e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.104 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097ffff700 1 -- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09800a0ed0 msgr2=0x7f0980137980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.104 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097ffff700 1 --2- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09800a0ed0 0x7f0980137980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.105 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097ffff700 1 -- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0980060240 msgr2=0x7f0980063e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.105 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097ffff700 1 --2- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0980060240 0x7f0980063e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.105 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097ffff700 1 -- 172.21.15.154:0/3429960831 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096800b180 con 0x7f09801333c0 2022-11-27T17:02:10.106 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f0976ffd700 1 --2- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09800a0ed0 0x7f0980137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:10.106 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097ffff700 1 --2- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09801333c0 0x7f0980137e70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f0978001250 tx=0x7f0978007db0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:10.106 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097d7fa700 1 -- 172.21.15.154:0/3429960831 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0978005bb0 con 0x7f09801333c0 2022-11-27T17:02:10.107 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097d7fa700 1 -- 172.21.15.154:0/3429960831 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978005d10 con 0x7f09801333c0 2022-11-27T17:02:10.107 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f0985dea700 1 -- 172.21.15.154:0/3429960831 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09800a52e0 con 0x7f09801333c0 2022-11-27T17:02:10.108 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f097d7fa700 1 -- 172.21.15.154:0/3429960831 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09780072c0 con 0x7f09801333c0 2022-11-27T17:02:10.108 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.084+0000 7f0985dea700 1 -- 172.21.15.154:0/3429960831 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09800634c0 con 0x7f09801333c0 2022-11-27T17:02:10.108 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.085+0000 7f097d7fa700 1 -- 172.21.15.154:0/3429960831 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f0978010070 con 0x7f09801333c0 2022-11-27T17:02:10.109 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.085+0000 7f0985dea700 1 -- 172.21.15.154:0/3429960831 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0980063b50 con 0x7f09801333c0 2022-11-27T17:02:10.109 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.085+0000 7f097d7fa700 1 -- 172.21.15.154:0/3429960831 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f097804a010 con 0x7f09801333c0 2022-11-27T17:02:10.110 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.088+0000 7f097d7fa700 1 -- 172.21.15.154:0/3429960831 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f0980063b50 con 0x7f09801333c0 2022-11-27T17:02:10.205 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.202+0000 7f0985dea700 1 -- 172.21.15.154:0/3429960831 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0980063b50 con 0x7f09801333c0 2022-11-27T17:02:10.205 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.202+0000 7f097d7fa700 1 -- 172.21.15.154:0/3429960831 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v14) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0980063b50 con 0x7f09801333c0 2022-11-27T17:02:10.206 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 8 2022-11-27T17:02:10.207 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.203+0000 7f0985dea700 1 -- 172.21.15.154:0/3429960831 >> v1:172.21.15.154:6816/104574 conn(0x7f0964038da0 legacy=0x7f096403b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.207 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.203+0000 7f0985dea700 1 -- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09801333c0 msgr2=0x7f0980137e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.207 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.203+0000 7f0985dea700 1 --2- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09801333c0 0x7f0980137e70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f0978001250 tx=0x7f0978007db0).stop 2022-11-27T17:02:10.208 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.203+0000 7f0985dea700 1 -- 172.21.15.154:0/3429960831 shutdown_connections 2022-11-27T17:02:10.208 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.203+0000 7f0985dea700 1 --2- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f09801333c0 0x7f0980137e70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.208 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.204+0000 7f0985dea700 1 --2- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0980060240 0x7f0980063e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.208 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.204+0000 7f0985dea700 1 --2- 172.21.15.154:0/3429960831 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f09800a0ed0 0x7f0980137980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.209 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.204+0000 7f0985dea700 1 -- 172.21.15.154:0/3429960831 >> 172.21.15.154:0/3429960831 conn(0x7f098005dfb0 msgr2=0x7f0980131ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:10.209 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.204+0000 7f0985dea700 1 -- 172.21.15.154:0/3429960831 shutdown_connections 2022-11-27T17:02:10.209 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.204+0000 7f0985dea700 1 -- 172.21.15.154:0/3429960831 wait complete. 2022-11-27T17:02:10.217 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T17:02:10.320 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.317+0000 7f3327b4f700 1 Processor -- start 2022-11-27T17:02:10.321 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.317+0000 7f3327b4f700 1 -- start start 2022-11-27T17:02:10.326 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f3327b4f700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f332005d430 0x7f332009aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.326 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f3327b4f700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3320099810 0x7f3320139d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.326 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f3327b4f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332013a200 0x7f332005ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.327 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f3327b4f700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332005d810 con 0x7f3320099810 2022-11-27T17:02:10.327 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f3327b4f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332009a550 con 0x7f332013a200 2022-11-27T17:02:10.327 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f3327b4f700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3320139640 con 0x7f332005d430 2022-11-27T17:02:10.328 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33250ea700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3320099810 0x7f3320139d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.328 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33260ec700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332013a200 0x7f332005ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.328 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33250ea700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3320099810 0x7f3320139d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49972/0 (socket says 172.21.15.154:49972) 2022-11-27T17:02:10.328 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33260ec700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332013a200 0x7f332005ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49320/0 (socket says 172.21.15.154:49320) 2022-11-27T17:02:10.329 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33250ea700 1 -- 172.21.15.154:0/3025923373 learned_addr learned my addr 172.21.15.154:0/3025923373 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:10.329 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33258eb700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f332005d430 0x7f332009aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.329 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33260ec700 1 -- 172.21.15.154:0/3025923373 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f332005d430 msgr2=0x7f332009aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:10.329 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33260ec700 1 --2- 172.21.15.154:0/3025923373 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f332005d430 0x7f332009aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.330 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33260ec700 1 -- 172.21.15.154:0/3025923373 >> v2:172.21.15.154:3300/0 conn(0x7f3320099810 msgr2=0x7f3320139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:10.330 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33260ec700 1 --2- 172.21.15.154:0/3025923373 >> v2:172.21.15.154:3300/0 conn(0x7f3320099810 0x7f3320139d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.330 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33260ec700 1 -- 172.21.15.154:0/3025923373 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33201398e0 con 0x7f332013a200 2022-11-27T17:02:10.330 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33250ea700 1 --2- 172.21.15.154:0/3025923373 >> v2:172.21.15.154:3300/0 conn(0x7f3320099810 0x7f3320139d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:10.331 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.318+0000 7f33260ec700 1 --2- 172.21.15.154:0/3025923373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332013a200 0x7f332005ba50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f33180032f0 tx=0x7f33180091d0).ready entity=mon.1 client_cookie=12233bc645c62f23 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:10.331 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f33248e9700 1 -- 172.21.15.154:0/3025923373 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f331800b600 con 0x7f332013a200 2022-11-27T17:02:10.331 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f33248e9700 1 -- 172.21.15.154:0/3025923373 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331800b760 con 0x7f332013a200 2022-11-27T17:02:10.332 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f33248e9700 1 -- 172.21.15.154:0/3025923373 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f331800b8d0 con 0x7f332013a200 2022-11-27T17:02:10.332 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f3327b4f700 1 -- 172.21.15.154:0/3025923373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332013a200 msgr2=0x7f332005ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.332 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f3327b4f700 1 --2- 172.21.15.154:0/3025923373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332013a200 0x7f332005ba50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f33180032f0 tx=0x7f33180091d0).stop 2022-11-27T17:02:10.332 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f3327b4f700 1 -- 172.21.15.154:0/3025923373 shutdown_connections 2022-11-27T17:02:10.333 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f3327b4f700 1 --2- 172.21.15.154:0/3025923373 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332013a200 0x7f332005ba50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.333 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f3327b4f700 1 --2- 172.21.15.154:0/3025923373 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f332005d430 0x7f332009aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.333 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f3327b4f700 1 --2- 172.21.15.154:0/3025923373 >> v2:172.21.15.154:3300/0 conn(0x7f3320099810 0x7f3320139d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.333 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f3327b4f700 1 -- 172.21.15.154:0/3025923373 >> 172.21.15.154:0/3025923373 conn(0x7f33200616e0 msgr2=0x7f332005cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:10.333 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f3327b4f700 1 -- 172.21.15.154:0/3025923373 shutdown_connections 2022-11-27T17:02:10.334 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.319+0000 7f3327b4f700 1 -- 172.21.15.154:0/3025923373 wait complete. 2022-11-27T17:02:10.334 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.320+0000 7f3327b4f700 1 Processor -- start 2022-11-27T17:02:10.334 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.320+0000 7f3327b4f700 1 -- start start 2022-11-27T17:02:10.335 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f3327b4f700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332005d430 0x7f3320094620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.335 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f3327b4f700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3320099810 0x7f3320148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.335 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f3327b4f700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3320094b10 0x7f3320149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:10.335 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f3327b4f700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33200964c0 con 0x7f3320099810 2022-11-27T17:02:10.336 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f3327b4f700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3320095c70 con 0x7f332005d430 2022-11-27T17:02:10.336 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f3327b4f700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3320094ff0 con 0x7f3320094b10 2022-11-27T17:02:10.336 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33250ea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3320099810 0x7f3320148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.336 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33260ec700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3320094b10 0x7f3320149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.336 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33250ea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3320099810 0x7f3320148c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49986/0 (socket says 172.21.15.154:49986) 2022-11-27T17:02:10.337 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33250ea700 1 -- 172.21.15.154:0/318480702 learned_addr learned my addr 172.21.15.154:0/318480702 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:10.337 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33258eb700 1 --2- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332005d430 0x7f3320094620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:10.337 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33258eb700 1 -- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3320094b10 msgr2=0x7f3320149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.337 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33258eb700 1 --2- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3320094b10 0x7f3320149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.338 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33258eb700 1 -- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3320099810 msgr2=0x7f3320148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.338 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33258eb700 1 --2- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3320099810 0x7f3320148c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.338 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33258eb700 1 -- 172.21.15.154:0/318480702 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c00b1e0 con 0x7f332005d430 2022-11-27T17:02:10.338 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33260ec700 1 --2- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3320094b10 0x7f3320149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:10.339 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f33258eb700 1 --2- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332005d430 0x7f3320094620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f331000c000 tx=0x7f3310006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:10.339 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f330effd700 1 -- 172.21.15.154:0/318480702 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3310010780 con 0x7f332005d430 2022-11-27T17:02:10.339 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f330effd700 1 -- 172.21.15.154:0/318480702 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33100108e0 con 0x7f332005d430 2022-11-27T17:02:10.340 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f3327b4f700 1 -- 172.21.15.154:0/318480702 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3318009470 con 0x7f332005d430 2022-11-27T17:02:10.340 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f330effd700 1 -- 172.21.15.154:0/318480702 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3310010c40 con 0x7f332005d430 2022-11-27T17:02:10.340 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.321+0000 7f3327b4f700 1 -- 172.21.15.154:0/318480702 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33200975b0 con 0x7f332005d430 2022-11-27T17:02:10.340 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.322+0000 7f330effd700 1 -- 172.21.15.154:0/318480702 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f331001b400 con 0x7f332005d430 2022-11-27T17:02:10.341 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.322+0000 7f330effd700 1 -- 172.21.15.154:0/318480702 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f331004a510 con 0x7f332005d430 2022-11-27T17:02:10.341 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.322+0000 7f3327b4f700 1 -- 172.21.15.154:0/318480702 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3320068320 con 0x7f332005d430 2022-11-27T17:02:10.342 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.326+0000 7f330effd700 1 -- 172.21.15.154:0/318480702 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f3310033190 con 0x7f332005d430 2022-11-27T17:02:10.474 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.471+0000 7f3327b4f700 1 -- 172.21.15.154:0/318480702 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3320057a60 con 0x7f332005d430 2022-11-27T17:02:10.475 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.472+0000 7f330effd700 1 -- 172.21.15.154:0/318480702 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+794 (secure 0 0 0) 0x7f3310032d80 con 0x7f332005d430 2022-11-27T17:02:10.475 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:02:10.475 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6817/376711555","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6817","nonce":376711555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[]} 2022-11-27T17:02:10.477 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.473+0000 7f3327b4f700 1 -- 172.21.15.154:0/318480702 >> v1:172.21.15.154:6816/104574 conn(0x7f3300038dd0 legacy=0x7f330003b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.477 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.473+0000 7f3327b4f700 1 -- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332005d430 msgr2=0x7f3320094620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:10.477 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.473+0000 7f3327b4f700 1 --2- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332005d430 0x7f3320094620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f331000c000 tx=0x7f3310006df0).stop 2022-11-27T17:02:10.478 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.473+0000 7f3327b4f700 1 -- 172.21.15.154:0/318480702 shutdown_connections 2022-11-27T17:02:10.478 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.473+0000 7f3327b4f700 1 --2- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3320094b10 0x7f3320149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.478 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.473+0000 7f3327b4f700 1 --2- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f332005d430 0x7f3320094620 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.478 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.473+0000 7f3327b4f700 1 --2- 172.21.15.154:0/318480702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3320099810 0x7f3320148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:10.478 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.473+0000 7f3327b4f700 1 -- 172.21.15.154:0/318480702 >> 172.21.15.154:0/318480702 conn(0x7f33200616e0 msgr2=0x7f332005dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:10.479 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.473+0000 7f3327b4f700 1 -- 172.21.15.154:0/318480702 shutdown_connections 2022-11-27T17:02:10.479 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:10.473+0000 7f3327b4f700 1 -- 172.21.15.154:0/318480702 wait complete. 2022-11-27T17:02:10.479 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 2 2022-11-27T17:02:12.487 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T17:02:12.610 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.601+0000 7f15ae0ea700 1 Processor -- start 2022-11-27T17:02:12.611 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.601+0000 7f15ae0ea700 1 -- start start 2022-11-27T17:02:12.611 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15ae0ea700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15a8137aa0 0x7f15a8137e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:12.611 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15ae0ea700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a813c980 0x7f15a813ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:12.612 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15ae0ea700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a80677e0 con 0x7f15a8066cb0 2022-11-27T17:02:12.612 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15ae0ea700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80666d0 con 0x7f15a8137aa0 2022-11-27T17:02:12.612 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15ae0ea700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8137620 con 0x7f15a813c980 2022-11-27T17:02:12.612 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a813c980 0x7f15a813ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a77fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15a8137aa0 0x7f15a8137e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a77fe700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15a8137aa0 0x7f15a8137e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50000/0 (socket says 172.21.15.154:50000) 2022-11-27T17:02:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a7fff700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a813c980 0x7f15a813ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49326/0 (socket says 172.21.15.154:49326) 2022-11-27T17:02:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a77fe700 1 -- 172.21.15.154:0/698873686 learned_addr learned my addr 172.21.15.154:0/698873686 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:12.614 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a6ffd700 1 --1- 172.21.15.154:0/698873686 >> v1:172.21.15.154:6789/0 conn(0x7f15a8066cb0 0x7f15a8065fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40610/0 (socket says 172.21.15.154:40610) 2022-11-27T17:02:12.614 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a7fff700 1 -- 172.21.15.154:0/698873686 >> v2:172.21.15.154:3300/0 conn(0x7f15a8137aa0 msgr2=0x7f15a8137e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:12.614 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a7fff700 1 --2- 172.21.15.154:0/698873686 >> v2:172.21.15.154:3300/0 conn(0x7f15a8137aa0 0x7f15a8137e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.614 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a7fff700 1 -- 172.21.15.154:0/698873686 >> v1:172.21.15.154:6789/0 conn(0x7f15a8066cb0 legacy=0x7f15a8065fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.615 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a67fc700 1 -- 172.21.15.154:0/698873686 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917130516 0 0) 0x7f15a80677e0 con 0x7f15a8066cb0 2022-11-27T17:02:12.615 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a7fff700 1 -- 172.21.15.154:0/698873686 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a8137880 con 0x7f15a813c980 2022-11-27T17:02:12.615 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.602+0000 7f15a77fe700 1 --2- 172.21.15.154:0/698873686 >> v2:172.21.15.154:3300/0 conn(0x7f15a8137aa0 0x7f15a8137e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:12.616 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15a7fff700 1 --2- 172.21.15.154:0/698873686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a813c980 0x7f15a813ed70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f15a00032f0 tx=0x7f15a000d670).ready entity=mon.1 client_cookie=f660ac0bafffae54 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:12.616 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15a67fc700 1 -- 172.21.15.154:0/698873686 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15a0012070 con 0x7f15a813c980 2022-11-27T17:02:12.616 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15a67fc700 1 -- 172.21.15.154:0/698873686 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a000fb80 con 0x7f15a813c980 2022-11-27T17:02:12.616 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15a67fc700 1 -- 172.21.15.154:0/698873686 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15a000de80 con 0x7f15a813c980 2022-11-27T17:02:12.617 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15ae0ea700 1 -- 172.21.15.154:0/698873686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a813c980 msgr2=0x7f15a813ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.617 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15ae0ea700 1 --2- 172.21.15.154:0/698873686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a813c980 0x7f15a813ed70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f15a00032f0 tx=0x7f15a000d670).stop 2022-11-27T17:02:12.617 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15ae0ea700 1 -- 172.21.15.154:0/698873686 shutdown_connections 2022-11-27T17:02:12.617 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15ae0ea700 1 --2- 172.21.15.154:0/698873686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a813c980 0x7f15a813ed70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.618 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15ae0ea700 1 --2- 172.21.15.154:0/698873686 >> v2:172.21.15.154:3300/0 conn(0x7f15a8137aa0 0x7f15a8137e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.618 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15ae0ea700 1 -- 172.21.15.154:0/698873686 >> 172.21.15.154:0/698873686 conn(0x7f15a805c4f0 msgr2=0x7f15a80a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:12.618 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15ae0ea700 1 -- 172.21.15.154:0/698873686 shutdown_connections 2022-11-27T17:02:12.619 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.603+0000 7f15ae0ea700 1 -- 172.21.15.154:0/698873686 wait complete. 2022-11-27T17:02:12.619 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.604+0000 7f15ae0ea700 1 Processor -- start 2022-11-27T17:02:12.619 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.605+0000 7f15ae0ea700 1 -- start start 2022-11-27T17:02:12.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.605+0000 7f15ae0ea700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f15a8066cb0 0x7f15a8139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:12.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.605+0000 7f15ae0ea700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8137aa0 0x7f15a8132430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:12.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.605+0000 7f15ae0ea700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a813c980 0x7f15a8132920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:12.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.605+0000 7f15ae0ea700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a813a3d0 con 0x7f15a813c980 2022-11-27T17:02:12.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.605+0000 7f15ae0ea700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8139c00 con 0x7f15a8137aa0 2022-11-27T17:02:12.621 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.605+0000 7f15ae0ea700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8139ec0 con 0x7f15a8066cb0 2022-11-27T17:02:12.621 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.605+0000 7f15a6ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8137aa0 0x7f15a8132430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:12.621 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.605+0000 7f15a6ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8137aa0 0x7f15a8132430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49334/0 (socket says 172.21.15.154:49334) 2022-11-27T17:02:12.621 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.605+0000 7f15a6ffd700 1 -- 172.21.15.154:0/370333209 learned_addr learned my addr 172.21.15.154:0/370333209 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:12.622 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a7fff700 1 --2- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a813c980 0x7f15a8132920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:12.622 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a77fe700 1 --2- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f15a8066cb0 0x7f15a8139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:12.622 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a6ffd700 1 -- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f15a8066cb0 msgr2=0x7f15a8139340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.622 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a6ffd700 1 --2- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f15a8066cb0 0x7f15a8139340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.623 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a6ffd700 1 -- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a813c980 msgr2=0x7f15a8132920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.623 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a6ffd700 1 --2- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a813c980 0x7f15a8132920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.623 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a6ffd700 1 -- 172.21.15.154:0/370333209 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c00b1e0 con 0x7f15a8137aa0 2022-11-27T17:02:12.624 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a6ffd700 1 --2- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8137aa0 0x7f15a8132430 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1598000bc0 tx=0x7f1598007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:12.624 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a4ff9700 1 -- 172.21.15.154:0/370333209 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15980070d0 con 0x7f15a8137aa0 2022-11-27T17:02:12.624 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a4ff9700 1 -- 172.21.15.154:0/370333209 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598007230 con 0x7f15a8137aa0 2022-11-27T17:02:12.624 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15ae0ea700 1 -- 172.21.15.154:0/370333209 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a000d910 con 0x7f15a8137aa0 2022-11-27T17:02:12.624 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15a4ff9700 1 -- 172.21.15.154:0/370333209 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1598005740 con 0x7f15a8137aa0 2022-11-27T17:02:12.625 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.606+0000 7f15ae0ea700 1 -- 172.21.15.154:0/370333209 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a80a8a10 con 0x7f15a8137aa0 2022-11-27T17:02:12.625 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.607+0000 7f15a4ff9700 1 -- 172.21.15.154:0/370333209 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f159801a060 con 0x7f15a8137aa0 2022-11-27T17:02:12.625 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.607+0000 7f15ae0ea700 1 -- 172.21.15.154:0/370333209 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a805cc90 con 0x7f15a8137aa0 2022-11-27T17:02:12.626 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.607+0000 7f15a4ff9700 1 -- 172.21.15.154:0/370333209 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f1598049930 con 0x7f15a8137aa0 2022-11-27T17:02:12.626 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.610+0000 7f15a4ff9700 1 -- 172.21.15.154:0/370333209 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f159802eb80 con 0x7f15a8137aa0 2022-11-27T17:02:12.754 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.751+0000 7f15ae0ea700 1 -- 172.21.15.154:0/370333209 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15a80a8500 con 0x7f15a8137aa0 2022-11-27T17:02:12.755 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.751+0000 7f15a4ff9700 1 -- 172.21.15.154:0/370333209 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+1378 (secure 0 0 0) 0x7f1598018030 con 0x7f15a8137aa0 2022-11-27T17:02:12.755 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:02:12.756 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6818/718201972","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6818","nonce":718201972}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6819/553878695","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6819","nonce":553878695}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6817/376711555","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6817","nonce":376711555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[]} 2022-11-27T17:02:12.757 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.752+0000 7f15ae0ea700 1 -- 172.21.15.154:0/370333209 >> v1:172.21.15.154:6816/104574 conn(0x7f1588038da0 legacy=0x7f158803b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.757 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.752+0000 7f15ae0ea700 1 -- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8137aa0 msgr2=0x7f15a8132430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.758 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.752+0000 7f15ae0ea700 1 --2- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8137aa0 0x7f15a8132430 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1598000bc0 tx=0x7f1598007a60).stop 2022-11-27T17:02:12.758 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.752+0000 7f15ae0ea700 1 -- 172.21.15.154:0/370333209 shutdown_connections 2022-11-27T17:02:12.758 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.752+0000 7f15ae0ea700 1 --2- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15a813c980 0x7f15a8132920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.758 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.752+0000 7f15ae0ea700 1 --2- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f15a8066cb0 0x7f15a8139340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.759 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.752+0000 7f15ae0ea700 1 --2- 172.21.15.154:0/370333209 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f15a8137aa0 0x7f15a8132430 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.759 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.752+0000 7f15ae0ea700 1 -- 172.21.15.154:0/370333209 >> 172.21.15.154:0/370333209 conn(0x7f15a805c4f0 msgr2=0x7f15a8136760 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:12.759 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.752+0000 7f15ae0ea700 1 -- 172.21.15.154:0/370333209 shutdown_connections 2022-11-27T17:02:12.759 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.753+0000 7f15ae0ea700 1 -- 172.21.15.154:0/370333209 wait complete. 2022-11-27T17:02:12.760 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 3 2022-11-27T17:02:12.768 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T17:02:12.892 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.869+0000 7fda1d709700 1 Processor -- start 2022-11-27T17:02:12.893 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.869+0000 7fda1d709700 1 -- start start 2022-11-27T17:02:12.893 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda1d709700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda18139aa0 0x7fda18139e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:12.894 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda1d709700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 0x7fda18141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:12.894 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda1d709700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda180a7430 con 0x7fda1805cb90 2022-11-27T17:02:12.895 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda1d709700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1805c690 con 0x7fda18060490 2022-11-27T17:02:12.895 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda1d709700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda1813a370 con 0x7fda18139aa0 2022-11-27T17:02:12.896 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda18139aa0 0x7fda18139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:12.896 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda177fe700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 0x7fda18141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:12.897 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda167fc700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fda1805cb90 0x7fda1805fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40622/0 (socket says 172.21.15.154:40622) 2022-11-27T17:02:12.897 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda167fc700 1 -- 172.21.15.154:0/4210194869 learned_addr learned my addr 172.21.15.154:0/4210194869 (peer_addr_for_me v1:172.21.15.154:0/0) 2022-11-27T17:02:12.897 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda15ffb700 1 -- 172.21.15.154:0/4210194869 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826749085 0 0) 0x7fda180a7430 con 0x7fda1805cb90 2022-11-27T17:02:12.898 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda16ffd700 1 -- 172.21.15.154:0/4210194869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 msgr2=0x7fda18141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:12.898 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda16ffd700 1 --2- 172.21.15.154:0/4210194869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 0x7fda18141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.899 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda16ffd700 1 -- 172.21.15.154:0/4210194869 >> v1:172.21.15.154:6789/0 conn(0x7fda1805cb90 legacy=0x7fda1805fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.900 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda16ffd700 1 -- 172.21.15.154:0/4210194869 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1805d350 con 0x7fda18139aa0 2022-11-27T17:02:12.900 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda177fe700 1 --2- 172.21.15.154:0/4210194869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 0x7fda18141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:12.900 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda16ffd700 1 --2- 172.21.15.154:0/4210194869 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda18139aa0 0x7fda18139e80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fda0c002f70 tx=0x7fda0c006df0).ready entity=mon.2 client_cookie=ddf853ee3cc1118 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:12.901 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.870+0000 7fda15ffb700 1 -- 172.21.15.154:0/4210194869 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda0c00d570 con 0x7fda18139aa0 2022-11-27T17:02:12.901 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.871+0000 7fda15ffb700 1 -- 172.21.15.154:0/4210194869 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c000e90 con 0x7fda18139aa0 2022-11-27T17:02:12.902 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.871+0000 7fda15ffb700 1 -- 172.21.15.154:0/4210194869 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda0c00d8c0 con 0x7fda18139aa0 2022-11-27T17:02:12.902 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.871+0000 7fda1d709700 1 -- 172.21.15.154:0/4210194869 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda18139aa0 msgr2=0x7fda18139e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.903 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.871+0000 7fda1d709700 1 --2- 172.21.15.154:0/4210194869 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda18139aa0 0x7fda18139e80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fda0c002f70 tx=0x7fda0c006df0).stop 2022-11-27T17:02:12.903 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.871+0000 7fda1d709700 1 -- 172.21.15.154:0/4210194869 shutdown_connections 2022-11-27T17:02:12.904 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.871+0000 7fda1d709700 1 --2- 172.21.15.154:0/4210194869 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 0x7fda18141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.904 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.871+0000 7fda1d709700 1 --2- 172.21.15.154:0/4210194869 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda18139aa0 0x7fda18139e80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.905 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.871+0000 7fda1d709700 1 -- 172.21.15.154:0/4210194869 >> 172.21.15.154:0/4210194869 conn(0x7fda180671b0 msgr2=0x7fda1805dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:12.905 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.871+0000 7fda1d709700 1 -- 172.21.15.154:0/4210194869 shutdown_connections 2022-11-27T17:02:12.906 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.871+0000 7fda1d709700 1 -- 172.21.15.154:0/4210194869 wait complete. 2022-11-27T17:02:12.906 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.872+0000 7fda1d709700 1 Processor -- start 2022-11-27T17:02:12.907 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.872+0000 7fda1d709700 1 -- start start 2022-11-27T17:02:12.907 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda1d709700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda1805cb90 0x7fda18115770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:12.908 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda1d709700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 0x7fda18148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:12.908 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda1d709700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda18139aa0 0x7fda18149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:12.909 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda1d709700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda18116830 con 0x7fda18139aa0 2022-11-27T17:02:12.909 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda1d709700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda181162f0 con 0x7fda18060490 2022-11-27T17:02:12.910 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda1d709700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fda18116030 con 0x7fda1805cb90 2022-11-27T17:02:12.910 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda1805cb90 0x7fda18115770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:12.911 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda16ffd700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda1805cb90 0x7fda18115770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55254/0 (socket says 172.21.15.154:55254) 2022-11-27T17:02:12.911 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda16ffd700 1 -- 172.21.15.154:0/392908274 learned_addr learned my addr 172.21.15.154:0/392908274 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:12.912 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda167fc700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 0x7fda18148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:12.912 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda177fe700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda18139aa0 0x7fda18149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:12.913 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda16ffd700 1 -- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 msgr2=0x7fda18148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.913 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda16ffd700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 0x7fda18148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.913 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda16ffd700 1 -- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda18139aa0 msgr2=0x7fda18149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.914 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda16ffd700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda18139aa0 0x7fda18149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.914 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda16ffd700 1 -- 172.21.15.154:0/392908274 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda10009470 con 0x7fda1805cb90 2022-11-27T17:02:12.915 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda177fe700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda18139aa0 0x7fda18149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T17:02:12.915 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda167fc700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 0x7fda18148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:12.915 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda16ffd700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda1805cb90 0x7fda18115770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fda0c002f70 tx=0x7fda0c014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:12.916 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda07fff700 1 -- 172.21.15.154:0/392908274 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda0c016070 con 0x7fda1805cb90 2022-11-27T17:02:12.916 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda07fff700 1 -- 172.21.15.154:0/392908274 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c00de90 con 0x7fda1805cb90 2022-11-27T17:02:12.916 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda1d709700 1 -- 172.21.15.154:0/392908274 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c00b180 con 0x7fda1805cb90 2022-11-27T17:02:12.917 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda07fff700 1 -- 172.21.15.154:0/392908274 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda0c013e50 con 0x7fda1805cb90 2022-11-27T17:02:12.917 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.873+0000 7fda1d709700 1 -- 172.21.15.154:0/392908274 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda180abc30 con 0x7fda1805cb90 2022-11-27T17:02:12.918 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.874+0000 7fda07fff700 1 -- 172.21.15.154:0/392908274 <== mon.2 v2:172.21.15.154:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fda0c025030 con 0x7fda1805cb90 2022-11-27T17:02:12.918 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.874+0000 7fda1d709700 1 -- 172.21.15.154:0/392908274 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda18114e20 con 0x7fda1805cb90 2022-11-27T17:02:12.918 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.874+0000 7fda07fff700 1 -- 172.21.15.154:0/392908274 <== mon.2 v2:172.21.15.154:3302/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fda0c04b600 con 0x7fda1805cb90 2022-11-27T17:02:12.919 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.877+0000 7fda07fff700 1 -- 172.21.15.154:0/392908274 <== mon.2 v2:172.21.15.154:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fda0c01fdb0 con 0x7fda1805cb90 2022-11-27T17:02:12.990 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.987+0000 7fda1d709700 1 -- 172.21.15.154:0/392908274 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda181395d0 con 0x7fda1805cb90 2022-11-27T17:02:12.992 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.988+0000 7fda07fff700 1 -- 172.21.15.154:0/392908274 <== mon.2 v2:172.21.15.154:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+7677 (secure 0 0 0) 0x7fda0c01b2b0 con 0x7fda1805cb90 2022-11-27T17:02:12.993 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:02:12.993 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":15,"fsid":"5adefcc8-72cd-433b-bd69-ec1eac0a6bf2","created":"2022-11-27T17:01:59.370943+0000","modified":"2022-11-27T17:02:10.447670+0000","last_up_change":"2022-11-27T17:02:05.307461+0000","last_in_change":"2022-11-27T17:02:05.307461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-11-27T17:02:01.324848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T17:02:08.341335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3061801a-e670-4a13-9a7d-6c526f88ef28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6808","nonce":105154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6809","nonce":105154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6811","nonce":105154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6810","nonce":105154}]},"public_addr":"172.21.15.154:6808/105154","cluster_addr":"172.21.15.154:6809/105154","heartbeat_back_addr":"172.21.15.154:6811/105154","heartbeat_front_addr":"172.21.15.154:6810/105154","state":["exists","up"]},{"osd":1,"uuid":"6906d403-7e5e-4ac7-9739-9178ecf56f2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6800","nonce":105155}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6801","nonce":105155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6803","nonce":105155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6802","nonce":105155}]},"public_addr":"172.21.15.154:6800/105155","cluster_addr":"172.21.15.154:6801/105155","heartbeat_back_addr":"172.21.15.154:6803/105155","heartbeat_front_addr":"172.21.15.154:6802/105155","state":["exists","up"]},{"osd":2,"uuid":"5bcff406-28bc-465d-8c76-56626ab7936a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6804","nonce":105156}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6805","nonce":105156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6807","nonce":105156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6806","nonce":105156}]},"public_addr":"172.21.15.154:6804/105156","cluster_addr":"172.21.15.154:6805/105156","heartbeat_back_addr":"172.21.15.154:6807/105156","heartbeat_front_addr":"172.21.15.154:6806/105156","state":["exists","up"]},{"osd":3,"uuid":"d6f7e6fc-d825-4e61-b433-6d5cf4075863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6812","nonce":105157}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6813","nonce":105157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6815","nonce":105157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6814","nonce":105157}]},"public_addr":"172.21.15.154:6812/105157","cluster_addr":"172.21.15.154:6813/105157","heartbeat_back_addr":"172.21.15.154:6815/105157","heartbeat_front_addr":"172.21.15.154:6814/105157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.393188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.290316+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.346843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.384664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T17:02:12.995 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.989+0000 7fda1d709700 1 -- 172.21.15.154:0/392908274 >> v1:172.21.15.154:6816/104574 conn(0x7fd9f4038da0 legacy=0x7fd9f403b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.995 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.989+0000 7fda1d709700 1 -- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda1805cb90 msgr2=0x7fda18115770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:12.996 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.989+0000 7fda1d709700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda1805cb90 0x7fda18115770 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fda0c002f70 tx=0x7fda0c014000).stop 2022-11-27T17:02:12.996 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.989+0000 7fda1d709700 1 -- 172.21.15.154:0/392908274 shutdown_connections 2022-11-27T17:02:12.997 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.989+0000 7fda1d709700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda18139aa0 0x7fda18149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.997 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.989+0000 7fda1d709700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fda1805cb90 0x7fda18115770 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.997 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.989+0000 7fda1d709700 1 --2- 172.21.15.154:0/392908274 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fda18060490 0x7fda18148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:12.998 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.989+0000 7fda1d709700 1 -- 172.21.15.154:0/392908274 >> 172.21.15.154:0/392908274 conn(0x7fda180671b0 msgr2=0x7fda180a1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:12.998 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.989+0000 7fda1d709700 1 -- 172.21.15.154:0/392908274 shutdown_connections 2022-11-27T17:02:12.999 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:12.989+0000 7fda1d709700 1 -- 172.21.15.154:0/392908274 wait complete. 2022-11-27T17:02:13.003 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-11-27T17:02:01.324848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T17:02:08.341335+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T17:02:13.004 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-11-27T17:02:13.109 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.106+0000 7fe514f35700 1 Processor -- start 2022-11-27T17:02:13.110 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.106+0000 7fe514f35700 1 -- start start 2022-11-27T17:02:13.116 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe514f35700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510094540 0x7fe510093410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.116 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe514f35700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe5101325e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.116 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe514f35700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe510132ad0 0x7fe5100a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.117 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe514f35700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe510093900 con 0x7fe510132ad0 2022-11-27T17:02:13.117 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe514f35700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe510094090 con 0x7fe5100922d0 2022-11-27T17:02:13.117 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe514f35700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe510132fb0 con 0x7fe510094540 2022-11-27T17:02:13.118 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe50dd9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe5101325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.119 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe50ed9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe510132ad0 0x7fe5100a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.119 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe50e59c700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510094540 0x7fe510093410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.119 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe50dd9b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe5101325e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49362/0 (socket says 172.21.15.154:49362) 2022-11-27T17:02:13.120 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe50ed9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe510132ad0 0x7fe5100a80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50014/0 (socket says 172.21.15.154:50014) 2022-11-27T17:02:13.120 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe50dd9b700 1 -- 172.21.15.154:0/1872983963 learned_addr learned my addr 172.21.15.154:0/1872983963 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:13.120 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.107+0000 7fe50e59c700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510094540 0x7fe510093410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55256/0 (socket says 172.21.15.154:55256) 2022-11-27T17:02:13.120 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe50ed9d700 1 -- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510094540 msgr2=0x7fe510093410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:13.120 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe50ed9d700 1 --2- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510094540 0x7fe510093410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.121 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe50ed9d700 1 -- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 msgr2=0x7fe5101325e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:13.121 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe50ed9d700 1 --2- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe5101325e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.121 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe50ed9d700 1 -- 172.21.15.154:0/1872983963 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe510094920 con 0x7fe510132ad0 2022-11-27T17:02:13.121 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe50dd9b700 1 --2- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe5101325e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:13.122 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe50ed9d700 1 --2- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe510132ad0 0x7fe5100a80f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fe510093a40 tx=0x7fe5080091d0).ready entity=mon.0 client_cookie=972c3c49d5a5b200 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:13.122 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe50d59a700 1 -- 172.21.15.154:0/1872983963 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50800b600 con 0x7fe510132ad0 2022-11-27T17:02:13.122 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe50d59a700 1 -- 172.21.15.154:0/1872983963 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50800b760 con 0x7fe510132ad0 2022-11-27T17:02:13.122 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe50d59a700 1 -- 172.21.15.154:0/1872983963 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50800b8d0 con 0x7fe510132ad0 2022-11-27T17:02:13.122 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe514f35700 1 -- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe510132ad0 msgr2=0x7fe5100a80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.123 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe514f35700 1 --2- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe510132ad0 0x7fe5100a80f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fe510093a40 tx=0x7fe5080091d0).stop 2022-11-27T17:02:13.123 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe514f35700 1 -- 172.21.15.154:0/1872983963 shutdown_connections 2022-11-27T17:02:13.123 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe514f35700 1 --2- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe510132ad0 0x7fe5100a80f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.123 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.108+0000 7fe514f35700 1 --2- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510094540 0x7fe510093410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.124 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.109+0000 7fe514f35700 1 --2- 172.21.15.154:0/1872983963 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe5101325e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.124 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.109+0000 7fe514f35700 1 -- 172.21.15.154:0/1872983963 >> 172.21.15.154:0/1872983963 conn(0x7fe51005ba50 msgr2=0x7fe51005be60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:13.124 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.109+0000 7fe514f35700 1 -- 172.21.15.154:0/1872983963 shutdown_connections 2022-11-27T17:02:13.124 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.109+0000 7fe514f35700 1 -- 172.21.15.154:0/1872983963 wait complete. 2022-11-27T17:02:13.125 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.109+0000 7fe514f35700 1 Processor -- start 2022-11-27T17:02:13.125 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.109+0000 7fe514f35700 1 -- start start 2022-11-27T17:02:13.125 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe514f35700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe510062620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.125 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe514f35700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe510094540 0x7fe510062dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.125 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe514f35700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510132ad0 0x7fe510064f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.126 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe514f35700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101357b0 con 0x7fe510094540 2022-11-27T17:02:13.126 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe514f35700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe510135a50 con 0x7fe5100922d0 2022-11-27T17:02:13.126 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe514f35700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe510135d10 con 0x7fe510132ad0 2022-11-27T17:02:13.126 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50e59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe510062620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.126 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50dd9b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe510094540 0x7fe510062dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50e59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe510062620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49370/0 (socket says 172.21.15.154:49370) 2022-11-27T17:02:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50e59c700 1 -- 172.21.15.154:0/157066686 learned_addr learned my addr 172.21.15.154:0/157066686 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50ed9d700 1 --2- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510132ad0 0x7fe510064f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.127 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50dd9b700 1 -- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510132ad0 msgr2=0x7fe510064f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.128 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50dd9b700 1 --2- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510132ad0 0x7fe510064f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.128 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50dd9b700 1 -- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 msgr2=0x7fe510062620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.128 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50dd9b700 1 --2- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe510062620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.128 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50dd9b700 1 -- 172.21.15.154:0/157066686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f800b2c0 con 0x7fe510094540 2022-11-27T17:02:13.128 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50e59c700 1 --2- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe510062620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:13.129 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.110+0000 7fe50dd9b700 1 --2- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe510094540 0x7fe510062dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe4f800c000 tx=0x7fe4f8006e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:13.129 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.111+0000 7fe5077fe700 1 -- 172.21.15.154:0/157066686 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4f8009040 con 0x7fe510094540 2022-11-27T17:02:13.129 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.111+0000 7fe5077fe700 1 -- 172.21.15.154:0/157066686 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f800c960 con 0x7fe510094540 2022-11-27T17:02:13.129 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.111+0000 7fe514f35700 1 -- 172.21.15.154:0/157066686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe508009470 con 0x7fe510094540 2022-11-27T17:02:13.129 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.111+0000 7fe5077fe700 1 -- 172.21.15.154:0/157066686 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4f800ccc0 con 0x7fe510094540 2022-11-27T17:02:13.130 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.111+0000 7fe514f35700 1 -- 172.21.15.154:0/157066686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe510065420 con 0x7fe510094540 2022-11-27T17:02:13.130 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.112+0000 7fe5077fe700 1 -- 172.21.15.154:0/157066686 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fe4f8021070 con 0x7fe510094540 2022-11-27T17:02:13.130 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.112+0000 7fe514f35700 1 -- 172.21.15.154:0/157066686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5100a85e0 con 0x7fe510094540 2022-11-27T17:02:13.130 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.112+0000 7fe5077fe700 1 -- 172.21.15.154:0/157066686 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fe4f804abd0 con 0x7fe510094540 2022-11-27T17:02:13.131 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.115+0000 7fe5077fe700 1 -- 172.21.15.154:0/157066686 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fe4f8033850 con 0x7fe510094540 2022-11-27T17:02:13.230 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.227+0000 7fe514f35700 1 -- 172.21.15.154:0/157066686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe5100920b0 con 0x7fe510094540 2022-11-27T17:02:13.231 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.227+0000 7fe5077fe700 1 -- 172.21.15.154:0/157066686 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v15) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe4f8033440 con 0x7fe510094540 2022-11-27T17:02:13.232 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2022-11-27T17:02:13.235 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.228+0000 7fe514f35700 1 -- 172.21.15.154:0/157066686 >> v1:172.21.15.154:6816/104574 conn(0x7fe4f4038dd0 legacy=0x7fe4f403b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.235 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.228+0000 7fe514f35700 1 -- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe510094540 msgr2=0x7fe510062dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.236 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.228+0000 7fe514f35700 1 --2- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe510094540 0x7fe510062dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fe4f800c000 tx=0x7fe4f8006e80).stop 2022-11-27T17:02:13.236 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.228+0000 7fe514f35700 1 -- 172.21.15.154:0/157066686 shutdown_connections 2022-11-27T17:02:13.237 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.228+0000 7fe514f35700 1 --2- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fe510132ad0 0x7fe510064f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.237 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.229+0000 7fe514f35700 1 --2- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe5100922d0 0x7fe510062620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.238 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.229+0000 7fe514f35700 1 --2- 172.21.15.154:0/157066686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe510094540 0x7fe510062dc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.238 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.229+0000 7fe514f35700 1 -- 172.21.15.154:0/157066686 >> 172.21.15.154:0/157066686 conn(0x7fe51005ba50 msgr2=0x7fe5100a6740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:13.238 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.229+0000 7fe514f35700 1 -- 172.21.15.154:0/157066686 shutdown_connections 2022-11-27T17:02:13.239 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.229+0000 7fe514f35700 1 -- 172.21.15.154:0/157066686 wait complete. 2022-11-27T17:02:13.242 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T17:02:13.350 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.347+0000 7f25e1029700 1 Processor -- start 2022-11-27T17:02:13.351 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.347+0000 7f25e1029700 1 -- start start 2022-11-27T17:02:13.362 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25e1029700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25dc139880 0x7f25dc099400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.363 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25e1029700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc139c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.363 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25e1029700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc13a150 0x7f25dc1330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.364 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25e1029700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25dc097f20 con 0x7f25dc139880 2022-11-27T17:02:13.364 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25e1029700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc099bd0 con 0x7f25dc0974a0 2022-11-27T17:02:13.365 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25e1029700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc097c80 con 0x7f25dc13a150 2022-11-27T17:02:13.365 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25da59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.366 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25db59e700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc13a150 0x7f25dc1330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.366 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25dad9d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25dc139880 0x7f25dc099400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.366 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25da59c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc139c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49382/0 (socket says 172.21.15.154:49382) 2022-11-27T17:02:13.367 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25da59c700 1 -- 172.21.15.154:0/1195384073 learned_addr learned my addr 172.21.15.154:0/1195384073 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:13.367 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25da59c700 1 -- 172.21.15.154:0/1195384073 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc13a150 msgr2=0x7f25dc1330f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:13.368 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25da59c700 1 --2- 172.21.15.154:0/1195384073 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc13a150 0x7f25dc1330f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.368 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25da59c700 1 -- 172.21.15.154:0/1195384073 >> v2:172.21.15.154:3300/0 conn(0x7f25dc139880 msgr2=0x7f25dc099400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:13.369 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25da59c700 1 --2- 172.21.15.154:0/1195384073 >> v2:172.21.15.154:3300/0 conn(0x7f25dc139880 0x7f25dc099400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.369 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25da59c700 1 -- 172.21.15.154:0/1195384073 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc05cb00 con 0x7f25dc0974a0 2022-11-27T17:02:13.370 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25db59e700 1 --2- 172.21.15.154:0/1195384073 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc13a150 0x7f25dc1330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:13.370 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25dad9d700 1 --2- 172.21.15.154:0/1195384073 >> v2:172.21.15.154:3300/0 conn(0x7f25dc139880 0x7f25dc099400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:13.371 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25da59c700 1 --2- 172.21.15.154:0/1195384073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc139c60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f25d0002f70 tx=0x7f25d0006eb0).ready entity=mon.1 client_cookie=ca123e8dbf4bdd2a server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:13.371 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25d9d9b700 1 -- 172.21.15.154:0/1195384073 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25d000d570 con 0x7f25dc0974a0 2022-11-27T17:02:13.372 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25d9d9b700 1 -- 172.21.15.154:0/1195384073 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d0000e90 con 0x7f25dc0974a0 2022-11-27T17:02:13.372 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.348+0000 7f25d9d9b700 1 -- 172.21.15.154:0/1195384073 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25d000d900 con 0x7f25dc0974a0 2022-11-27T17:02:13.372 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.349+0000 7f25e1029700 1 -- 172.21.15.154:0/1195384073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 msgr2=0x7f25dc139c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.373 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.349+0000 7f25e1029700 1 --2- 172.21.15.154:0/1195384073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc139c60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f25d0002f70 tx=0x7f25d0006eb0).stop 2022-11-27T17:02:13.373 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.349+0000 7f25e1029700 1 -- 172.21.15.154:0/1195384073 shutdown_connections 2022-11-27T17:02:13.374 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.349+0000 7f25e1029700 1 --2- 172.21.15.154:0/1195384073 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc13a150 0x7f25dc1330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.374 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.349+0000 7f25e1029700 1 --2- 172.21.15.154:0/1195384073 >> v2:172.21.15.154:3300/0 conn(0x7f25dc139880 0x7f25dc099400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.375 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.349+0000 7f25e1029700 1 --2- 172.21.15.154:0/1195384073 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc139c60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.375 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.349+0000 7f25e1029700 1 -- 172.21.15.154:0/1195384073 >> 172.21.15.154:0/1195384073 conn(0x7f25dc05c4f0 msgr2=0x7f25dc0a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:13.376 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.349+0000 7f25e1029700 1 -- 172.21.15.154:0/1195384073 shutdown_connections 2022-11-27T17:02:13.376 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.349+0000 7f25e1029700 1 -- 172.21.15.154:0/1195384073 wait complete. 2022-11-27T17:02:13.377 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.350+0000 7f25e1029700 1 Processor -- start 2022-11-27T17:02:13.377 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.350+0000 7f25e1029700 1 -- start start 2022-11-27T17:02:13.377 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25e1029700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc066ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.378 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25e1029700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc139880 0x7f25dc065380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.378 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25e1029700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25dc13a150 0x7f25dc1344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25e1029700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc067a10 con 0x7f25dc13a150 2022-11-27T17:02:13.379 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25e1029700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc067450 con 0x7f25dc0974a0 2022-11-27T17:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25e1029700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc0666c0 con 0x7f25dc139880 2022-11-27T17:02:13.380 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25dad9d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc066ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.381 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25dad9d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc066ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49384/0 (socket says 172.21.15.154:49384) 2022-11-27T17:02:13.381 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25dad9d700 1 -- 172.21.15.154:0/47488167 learned_addr learned my addr 172.21.15.154:0/47488167 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:13.382 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25db59e700 1 --2- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25dc13a150 0x7f25dc1344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.382 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25da59c700 1 --2- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc139880 0x7f25dc065380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.382 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25dad9d700 1 -- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc139880 msgr2=0x7f25dc065380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.383 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25dad9d700 1 --2- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc139880 0x7f25dc065380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.383 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25dad9d700 1 -- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25dc13a150 msgr2=0x7f25dc1344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.383 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25dad9d700 1 --2- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25dc13a150 0x7f25dc1344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25dad9d700 1 -- 172.21.15.154:0/47488167 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc00b180 con 0x7f25dc0974a0 2022-11-27T17:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25dad9d700 1 --2- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc066ea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f25dc066df0 tx=0x7f25cc006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25cbfff700 1 -- 172.21.15.154:0/47488167 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25cc00c850 con 0x7f25dc0974a0 2022-11-27T17:02:13.384 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25cbfff700 1 -- 172.21.15.154:0/47488167 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc00c9b0 con 0x7f25dc0974a0 2022-11-27T17:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25e1029700 1 -- 172.21.15.154:0/47488167 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d4009470 con 0x7f25dc0974a0 2022-11-27T17:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25cbfff700 1 -- 172.21.15.154:0/47488167 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f25cc00cd10 con 0x7f25dc0974a0 2022-11-27T17:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.351+0000 7f25e1029700 1 -- 172.21.15.154:0/47488167 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d000b1e0 con 0x7f25dc0974a0 2022-11-27T17:02:13.385 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.352+0000 7f25cbfff700 1 -- 172.21.15.154:0/47488167 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f25cc01c460 con 0x7f25dc0974a0 2022-11-27T17:02:13.386 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.352+0000 7f25e1029700 1 -- 172.21.15.154:0/47488167 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25dc13a5e0 con 0x7f25dc0974a0 2022-11-27T17:02:13.386 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.353+0000 7f25cbfff700 1 -- 172.21.15.154:0/47488167 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f25cc04ab80 con 0x7f25dc0974a0 2022-11-27T17:02:13.386 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.356+0000 7f25cbfff700 1 -- 172.21.15.154:0/47488167 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f25cc02fc20 con 0x7f25dc0974a0 2022-11-27T17:02:13.469 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.466+0000 7f25e1029700 1 -- 172.21.15.154:0/47488167 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f25dc062240 con 0x7f25dc0974a0 2022-11-27T17:02:13.470 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.466+0000 7f25cbfff700 1 -- 172.21.15.154:0/47488167 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f25cc0333f0 con 0x7f25dc0974a0 2022-11-27T17:02:13.471 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 8 2022-11-27T17:02:13.473 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.468+0000 7f25e1029700 1 -- 172.21.15.154:0/47488167 >> v1:172.21.15.154:6816/104574 conn(0x7f25b8038ec0 legacy=0x7f25b803b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.473 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.468+0000 7f25e1029700 1 -- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 msgr2=0x7f25dc066ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.473 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.468+0000 7f25e1029700 1 --2- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc066ea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f25dc066df0 tx=0x7f25cc006df0).stop 2022-11-27T17:02:13.473 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.468+0000 7f25e1029700 1 -- 172.21.15.154:0/47488167 shutdown_connections 2022-11-27T17:02:13.474 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.468+0000 7f25e1029700 1 --2- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25dc13a150 0x7f25dc1344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.474 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.468+0000 7f25e1029700 1 --2- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f25dc0974a0 0x7f25dc066ea0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.474 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.468+0000 7f25e1029700 1 --2- 172.21.15.154:0/47488167 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f25dc139880 0x7f25dc065380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.474 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.468+0000 7f25e1029700 1 -- 172.21.15.154:0/47488167 >> 172.21.15.154:0/47488167 conn(0x7f25dc05c4f0 msgr2=0x7f25dc136bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:13.475 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.468+0000 7f25e1029700 1 -- 172.21.15.154:0/47488167 shutdown_connections 2022-11-27T17:02:13.475 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.468+0000 7f25e1029700 1 -- 172.21.15.154:0/47488167 wait complete. 2022-11-27T17:02:13.481 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-11-27T17:02:13.592 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.582+0000 7fba475cc700 1 Processor -- start 2022-11-27T17:02:13.593 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.582+0000 7fba475cc700 1 -- start start 2022-11-27T17:02:13.593 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.582+0000 7fba475cc700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40139aa0 0x7fba40139e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.593 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.582+0000 7fba475cc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba4005cb90 0x7fba4005fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.594 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.582+0000 7fba475cc700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba400a7430 con 0x7fba40060270 2022-11-27T17:02:13.594 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.582+0000 7fba475cc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4005c690 con 0x7fba4005cb90 2022-11-27T17:02:13.594 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.582+0000 7fba475cc700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba4013a370 con 0x7fba40139aa0 2022-11-27T17:02:13.594 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba44b67700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba4005cb90 0x7fba4005fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.595 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba45b69700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fba40060270 0x7fba40140800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40636/0 (socket says 172.21.15.154:40636) 2022-11-27T17:02:13.595 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba45b69700 1 -- 172.21.15.154:0/874796210 learned_addr learned my addr 172.21.15.154:0/874796210 (peer_addr_for_me v1:172.21.15.154:0/0) 2022-11-27T17:02:13.595 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba45368700 1 --2- 172.21.15.154:0/874796210 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40139aa0 0x7fba40139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.596 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba2ffff700 1 -- 172.21.15.154:0/874796210 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826833192 0 0) 0x7fba400a7430 con 0x7fba40060270 2022-11-27T17:02:13.596 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba2ffff700 1 -- 172.21.15.154:0/874796210 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba28003260 con 0x7fba40060270 2022-11-27T17:02:13.596 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba44b67700 1 -- 172.21.15.154:0/874796210 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40139aa0 msgr2=0x7fba40139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:13.597 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba44b67700 1 --2- 172.21.15.154:0/874796210 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40139aa0 0x7fba40139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.597 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba44b67700 1 -- 172.21.15.154:0/874796210 >> v1:172.21.15.154:6789/0 conn(0x7fba40060270 legacy=0x7fba40140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.597 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba44b67700 1 -- 172.21.15.154:0/874796210 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4005d350 con 0x7fba4005cb90 2022-11-27T17:02:13.597 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba45368700 1 --2- 172.21.15.154:0/874796210 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40139aa0 0x7fba40139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:13.598 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba44b67700 1 --2- 172.21.15.154:0/874796210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba4005cb90 0x7fba4005fd80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fba4013a500 tx=0x7fba3c006df0).ready entity=mon.1 client_cookie=b21cd7203e6e94c9 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:13.598 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba2ffff700 1 -- 172.21.15.154:0/874796210 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba3c00d570 con 0x7fba4005cb90 2022-11-27T17:02:13.598 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba2ffff700 1 -- 172.21.15.154:0/874796210 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c000e90 con 0x7fba4005cb90 2022-11-27T17:02:13.598 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.583+0000 7fba2ffff700 1 -- 172.21.15.154:0/874796210 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba3c00d8c0 con 0x7fba4005cb90 2022-11-27T17:02:13.598 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.584+0000 7fba475cc700 1 -- 172.21.15.154:0/874796210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba4005cb90 msgr2=0x7fba4005fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.599 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.584+0000 7fba475cc700 1 --2- 172.21.15.154:0/874796210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba4005cb90 0x7fba4005fd80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fba4013a500 tx=0x7fba3c006df0).stop 2022-11-27T17:02:13.599 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.584+0000 7fba475cc700 1 -- 172.21.15.154:0/874796210 shutdown_connections 2022-11-27T17:02:13.599 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.584+0000 7fba475cc700 1 --2- 172.21.15.154:0/874796210 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40139aa0 0x7fba40139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.599 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.584+0000 7fba475cc700 1 --2- 172.21.15.154:0/874796210 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba4005cb90 0x7fba4005fd80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.600 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.584+0000 7fba475cc700 1 -- 172.21.15.154:0/874796210 >> 172.21.15.154:0/874796210 conn(0x7fba400671b0 msgr2=0x7fba4005dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:13.600 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.584+0000 7fba475cc700 1 -- 172.21.15.154:0/874796210 shutdown_connections 2022-11-27T17:02:13.600 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.584+0000 7fba475cc700 1 -- 172.21.15.154:0/874796210 wait complete. 2022-11-27T17:02:13.600 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba475cc700 1 Processor -- start 2022-11-27T17:02:13.600 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba475cc700 1 -- start start 2022-11-27T17:02:13.601 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba475cc700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba4005cb90 0x7fba401160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.601 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba475cc700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40060270 0x7fba40117290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.601 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba475cc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba40139aa0 0x7fba40137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.601 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba475cc700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba40116860 con 0x7fba4005cb90 2022-11-27T17:02:13.602 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba475cc700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba40114540 con 0x7fba40139aa0 2022-11-27T17:02:13.602 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba475cc700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba4013bfd0 con 0x7fba40060270 2022-11-27T17:02:13.602 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba45368700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba4005cb90 0x7fba401160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.602 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba45368700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba4005cb90 0x7fba401160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50054/0 (socket says 172.21.15.154:50054) 2022-11-27T17:02:13.603 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba45b69700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba40139aa0 0x7fba40137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.603 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba45368700 1 -- 172.21.15.154:0/1718814568 learned_addr learned my addr 172.21.15.154:0/1718814568 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:13.603 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.585+0000 7fba44b67700 1 --2- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40060270 0x7fba40117290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.603 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba45368700 1 -- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40060270 msgr2=0x7fba40117290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.604 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba45368700 1 --2- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40060270 0x7fba40117290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.604 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba45368700 1 -- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba40139aa0 msgr2=0x7fba40137130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.604 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba45368700 1 --2- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba40139aa0 0x7fba40137130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.604 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba45368700 1 -- 172.21.15.154:0/1718814568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3000b180 con 0x7fba4005cb90 2022-11-27T17:02:13.604 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba45b69700 1 --2- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba40139aa0 0x7fba40137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:13.605 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba45368700 1 --2- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba4005cb90 0x7fba401160d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fba3000c000 tx=0x7fba30006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:13.605 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba2e7fc700 1 -- 172.21.15.154:0/1718814568 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba30010780 con 0x7fba4005cb90 2022-11-27T17:02:13.605 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba2e7fc700 1 -- 172.21.15.154:0/1718814568 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba300108e0 con 0x7fba4005cb90 2022-11-27T17:02:13.605 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba475cc700 1 -- 172.21.15.154:0/1718814568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c00b180 con 0x7fba4005cb90 2022-11-27T17:02:13.606 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba2e7fc700 1 -- 172.21.15.154:0/1718814568 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba30010c40 con 0x7fba4005cb90 2022-11-27T17:02:13.606 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.586+0000 7fba475cc700 1 -- 172.21.15.154:0/1718814568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba40116b30 con 0x7fba4005cb90 2022-11-27T17:02:13.606 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.587+0000 7fba2e7fc700 1 -- 172.21.15.154:0/1718814568 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fba3001c460 con 0x7fba4005cb90 2022-11-27T17:02:13.607 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.587+0000 7fba2e7fc700 1 -- 172.21.15.154:0/1718814568 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fba3004a9e0 con 0x7fba4005cb90 2022-11-27T17:02:13.607 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.587+0000 7fba475cc700 1 -- 172.21.15.154:0/1718814568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba401395d0 con 0x7fba4005cb90 2022-11-27T17:02:13.607 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.590+0000 7fba2e7fc700 1 -- 172.21.15.154:0/1718814568 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fba30033660 con 0x7fba4005cb90 2022-11-27T17:02:13.705 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.702+0000 7fba475cc700 1 -- 172.21.15.154:0/1718814568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fba40057a60 con 0x7fba4005cb90 2022-11-27T17:02:13.706 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.702+0000 7fba2e7fc700 1 -- 172.21.15.154:0/1718814568 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v15) v1 ==== 84+0+143 (secure 0 0 0) 0x7fba3002b090 con 0x7fba4005cb90 2022-11-27T17:02:13.707 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:02:13.708 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2022-11-27T17:02:13.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2022-11-27T17:02:13.709 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2022-11-27T17:02:13.709 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": "device_health_metrics" 2022-11-27T17:02:13.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2022-11-27T17:02:13.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2022-11-27T17:02:13.711 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 2, 2022-11-27T17:02:13.712 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": "rbd" 2022-11-27T17:02:13.712 INFO:teuthology.orchestra.run.smithi154.stdout: } 2022-11-27T17:02:13.712 INFO:teuthology.orchestra.run.smithi154.stdout:] 2022-11-27T17:02:13.714 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.704+0000 7fba475cc700 1 -- 172.21.15.154:0/1718814568 >> v1:172.21.15.154:6816/104574 conn(0x7fba20038da0 legacy=0x7fba2003b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.715 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.704+0000 7fba475cc700 1 -- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba4005cb90 msgr2=0x7fba401160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.715 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.704+0000 7fba475cc700 1 --2- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba4005cb90 0x7fba401160d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fba3000c000 tx=0x7fba30006df0).stop 2022-11-27T17:02:13.716 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.704+0000 7fba475cc700 1 -- 172.21.15.154:0/1718814568 shutdown_connections 2022-11-27T17:02:13.716 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.704+0000 7fba475cc700 1 --2- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fba40139aa0 0x7fba40137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.717 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.704+0000 7fba475cc700 1 --2- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba4005cb90 0x7fba401160d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.717 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.704+0000 7fba475cc700 1 --2- 172.21.15.154:0/1718814568 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fba40060270 0x7fba40117290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.718 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.704+0000 7fba475cc700 1 -- 172.21.15.154:0/1718814568 >> 172.21.15.154:0/1718814568 conn(0x7fba400671b0 msgr2=0x7fba40138aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:13.718 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.704+0000 7fba475cc700 1 -- 172.21.15.154:0/1718814568 shutdown_connections 2022-11-27T17:02:13.718 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.704+0000 7fba475cc700 1 -- 172.21.15.154:0/1718814568 wait complete. 2022-11-27T17:02:13.721 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-11-27T17:02:13.722 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2022-11-27T17:02:13.830 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.827+0000 7f55edb2b700 1 Processor -- start 2022-11-27T17:02:13.831 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.827+0000 7f55edb2b700 1 -- start start 2022-11-27T17:02:13.836 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55edb2b700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e8139880 0x7f55e8099400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.836 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55edb2b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e80974a0 0x7f55e8139c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.837 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55edb2b700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e8099bd0 con 0x7f55e813a150 2022-11-27T17:02:13.837 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55edb2b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8097f20 con 0x7f55e80974a0 2022-11-27T17:02:13.837 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55edb2b700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8097c80 con 0x7f55e8139880 2022-11-27T17:02:13.837 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e80974a0 0x7f55e8139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.837 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e77fe700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e8139880 0x7f55e8099400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.838 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e80974a0 0x7f55e8139c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49416/0 (socket says 172.21.15.154:49416) 2022-11-27T17:02:13.838 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e77fe700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e8139880 0x7f55e8099400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55308/0 (socket says 172.21.15.154:55308) 2022-11-27T17:02:13.838 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e6ffd700 1 -- 172.21.15.154:0/2467978149 learned_addr learned my addr 172.21.15.154:0/2467978149 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:13.838 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e7fff700 1 --1- 172.21.15.154:0/2467978149 >> v1:172.21.15.154:6789/0 conn(0x7f55e813a150 0x7f55e81330f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40638/0 (socket says 172.21.15.154:40638) 2022-11-27T17:02:13.839 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e67fc700 1 -- 172.21.15.154:0/2467978149 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829099737 0 0) 0x7f55e8099bd0 con 0x7f55e813a150 2022-11-27T17:02:13.839 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e77fe700 1 -- 172.21.15.154:0/2467978149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e80974a0 msgr2=0x7f55e8139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:13.839 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e77fe700 1 --2- 172.21.15.154:0/2467978149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e80974a0 0x7f55e8139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.839 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e77fe700 1 -- 172.21.15.154:0/2467978149 >> v1:172.21.15.154:6789/0 conn(0x7f55e813a150 legacy=0x7f55e81330f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.840 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e77fe700 1 -- 172.21.15.154:0/2467978149 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e805cb00 con 0x7f55e8139880 2022-11-27T17:02:13.840 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.828+0000 7f55e6ffd700 1 --2- 172.21.15.154:0/2467978149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e80974a0 0x7f55e8139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:13.840 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55e77fe700 1 --2- 172.21.15.154:0/2467978149 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e8139880 0x7f55e8099400 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f55dc002f70 tx=0x7f55dc006df0).ready entity=mon.2 client_cookie=7239dc512078dfd3 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:13.840 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55e67fc700 1 -- 172.21.15.154:0/2467978149 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55dc00d570 con 0x7f55e8139880 2022-11-27T17:02:13.841 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55e67fc700 1 -- 172.21.15.154:0/2467978149 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc000e90 con 0x7f55e8139880 2022-11-27T17:02:13.841 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55e67fc700 1 -- 172.21.15.154:0/2467978149 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55dc00d8c0 con 0x7f55e8139880 2022-11-27T17:02:13.841 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55edb2b700 1 -- 172.21.15.154:0/2467978149 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e8139880 msgr2=0x7f55e8099400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.841 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55edb2b700 1 --2- 172.21.15.154:0/2467978149 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e8139880 0x7f55e8099400 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f55dc002f70 tx=0x7f55dc006df0).stop 2022-11-27T17:02:13.841 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55edb2b700 1 -- 172.21.15.154:0/2467978149 shutdown_connections 2022-11-27T17:02:13.842 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55edb2b700 1 --2- 172.21.15.154:0/2467978149 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e8139880 0x7f55e8099400 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.842 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55edb2b700 1 --2- 172.21.15.154:0/2467978149 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e80974a0 0x7f55e8139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.842 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55edb2b700 1 -- 172.21.15.154:0/2467978149 >> 172.21.15.154:0/2467978149 conn(0x7f55e805c4f0 msgr2=0x7f55e80a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:13.842 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55edb2b700 1 -- 172.21.15.154:0/2467978149 shutdown_connections 2022-11-27T17:02:13.843 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.829+0000 7f55edb2b700 1 -- 172.21.15.154:0/2467978149 wait complete. 2022-11-27T17:02:13.843 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.830+0000 7f55edb2b700 1 Processor -- start 2022-11-27T17:02:13.843 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.830+0000 7f55edb2b700 1 -- start start 2022-11-27T17:02:13.843 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55edb2b700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55e80974a0 0x7f55e8064c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.844 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55edb2b700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e8139880 0x7f55e8148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.844 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55edb2b700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e813a150 0x7f55e8149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:13.844 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55edb2b700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80655f0 con 0x7f55e80974a0 2022-11-27T17:02:13.844 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55edb2b700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80662c0 con 0x7f55e8139880 2022-11-27T17:02:13.844 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55edb2b700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8065d30 con 0x7f55e813a150 2022-11-27T17:02:13.845 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55e80974a0 0x7f55e8064c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.845 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e6ffd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e8139880 0x7f55e8148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.845 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e77fe700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55e80974a0 0x7f55e8064c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50058/0 (socket says 172.21.15.154:50058) 2022-11-27T17:02:13.845 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e77fe700 1 -- 172.21.15.154:0/2662290292 learned_addr learned my addr 172.21.15.154:0/2662290292 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:13.846 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e7fff700 1 --2- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e813a150 0x7f55e8149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:13.846 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e77fe700 1 -- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e813a150 msgr2=0x7f55e8149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.846 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e77fe700 1 --2- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e813a150 0x7f55e8149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.846 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e77fe700 1 -- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e8139880 msgr2=0x7f55e8148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:13.846 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e77fe700 1 --2- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e8139880 0x7f55e8148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:13.847 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e77fe700 1 -- 172.21.15.154:0/2662290292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d800b1e0 con 0x7f55e80974a0 2022-11-27T17:02:13.847 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e6ffd700 1 --2- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e8139880 0x7f55e8148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:13.847 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e77fe700 1 --2- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55e80974a0 0x7f55e8064c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f55dc002f70 tx=0x7f55dc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:13.847 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e4ff9700 1 -- 172.21.15.154:0/2662290292 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55dc013de0 con 0x7f55e80974a0 2022-11-27T17:02:13.848 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55e4ff9700 1 -- 172.21.15.154:0/2662290292 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc00de40 con 0x7f55e80974a0 2022-11-27T17:02:13.848 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.831+0000 7f55edb2b700 1 -- 172.21.15.154:0/2662290292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc00b180 con 0x7f55e80974a0 2022-11-27T17:02:13.848 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.832+0000 7f55e4ff9700 1 -- 172.21.15.154:0/2662290292 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55dc01e5f0 con 0x7f55e80974a0 2022-11-27T17:02:13.848 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.832+0000 7f55edb2b700 1 -- 172.21.15.154:0/2662290292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e80676b0 con 0x7f55e80974a0 2022-11-27T17:02:13.849 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.832+0000 7f55edb2b700 1 -- 172.21.15.154:0/2662290292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e8064860 con 0x7f55e80974a0 2022-11-27T17:02:13.849 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.833+0000 7f55e4ff9700 1 -- 172.21.15.154:0/2662290292 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f55dc025030 con 0x7f55e80974a0 2022-11-27T17:02:13.849 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.833+0000 7f55e4ff9700 1 -- 172.21.15.154:0/2662290292 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f55dc04b310 con 0x7f55e80974a0 2022-11-27T17:02:13.850 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.836+0000 7f55e4ff9700 1 -- 172.21.15.154:0/2662290292 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f55dc0058c0 con 0x7f55e80974a0 2022-11-27T17:02:13.951 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:13.948+0000 7f55edb2b700 1 -- 172.21.15.154:0/2662290292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f55e8064650 con 0x7f55e80974a0 2022-11-27T17:02:14.819 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.816+0000 7f55e4ff9700 1 -- 172.21.15.154:0/2662290292 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v16) v1 ==== 138+0+0 (secure 0 0 0) 0x7f55dc023030 con 0x7f55e80974a0 2022-11-27T17:02:14.822 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.820+0000 7f55edb2b700 1 -- 172.21.15.154:0/2662290292 >> v1:172.21.15.154:6816/104574 conn(0x7f55c8038da0 legacy=0x7f55c803b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.820+0000 7f55edb2b700 1 -- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55e80974a0 msgr2=0x7f55e8064c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.820+0000 7f55edb2b700 1 --2- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55e80974a0 0x7f55e8064c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f55dc002f70 tx=0x7f55dc014000).stop 2022-11-27T17:02:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.821+0000 7f55edb2b700 1 -- 172.21.15.154:0/2662290292 shutdown_connections 2022-11-27T17:02:14.824 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.821+0000 7f55edb2b700 1 --2- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f55e813a150 0x7f55e8149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:14.824 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.821+0000 7f55edb2b700 1 --2- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55e80974a0 0x7f55e8064c60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:14.824 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.821+0000 7f55edb2b700 1 --2- 172.21.15.154:0/2662290292 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f55e8139880 0x7f55e8148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:14.824 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.821+0000 7f55edb2b700 1 -- 172.21.15.154:0/2662290292 >> 172.21.15.154:0/2662290292 conn(0x7f55e805c4f0 msgr2=0x7f55e8136bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:14.827 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.825+0000 7f55edb2b700 1 -- 172.21.15.154:0/2662290292 shutdown_connections 2022-11-27T17:02:14.828 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.825+0000 7f55edb2b700 1 -- 172.21.15.154:0/2662290292 wait complete. 2022-11-27T17:02:14.828 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2022-11-27T17:02:14.840 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2022-11-27T17:02:14.956 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.953+0000 7f099f302700 1 Processor -- start 2022-11-27T17:02:14.956 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.953+0000 7f099f302700 1 -- start start 2022-11-27T17:02:14.962 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099f302700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0998136690 0x7f0998136a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:14.962 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099f302700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f099809b540 0x7f099809a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:14.963 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099f302700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0998067c60 0x7f099805ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:14.963 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099f302700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099809a500 con 0x7f0998067c60 2022-11-27T17:02:14.963 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099f302700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099809ae80 con 0x7f0998136690 2022-11-27T17:02:14.963 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099f302700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0998135fc0 con 0x7f099809b540 2022-11-27T17:02:14.964 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099d09e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0998136690 0x7f0998136a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:14.964 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099c89d700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f099809b540 0x7f099809a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:14.964 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099c89d700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f099809b540 0x7f099809a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55334/0 (socket says 172.21.15.154:55334) 2022-11-27T17:02:14.964 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099c89d700 1 -- 172.21.15.154:0/1679282799 learned_addr learned my addr 172.21.15.154:0/1679282799 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:14.965 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099d09e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0998136690 0x7f0998136a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49428/0 (socket says 172.21.15.154:49428) 2022-11-27T17:02:14.965 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099d89f700 1 --2- 172.21.15.154:0/1679282799 >> v2:172.21.15.154:3300/0 conn(0x7f0998067c60 0x7f099805ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:14.965 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099d09e700 1 -- 172.21.15.154:0/1679282799 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f099809b540 msgr2=0x7f099809a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:14.965 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099d09e700 1 --2- 172.21.15.154:0/1679282799 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f099809b540 0x7f099809a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:14.966 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099d09e700 1 -- 172.21.15.154:0/1679282799 >> v2:172.21.15.154:3300/0 conn(0x7f0998067c60 msgr2=0x7f099805ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:14.966 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099d09e700 1 --2- 172.21.15.154:0/1679282799 >> v2:172.21.15.154:3300/0 conn(0x7f0998067c60 0x7f099805ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:14.966 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099d09e700 1 -- 172.21.15.154:0/1679282799 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0998136260 con 0x7f0998136690 2022-11-27T17:02:14.966 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099c89d700 1 --2- 172.21.15.154:0/1679282799 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f099809b540 0x7f099809a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:14.967 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.954+0000 7f099d09e700 1 --2- 172.21.15.154:0/1679282799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0998136690 0x7f0998136a70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f099809a3b0 tx=0x7f0988006df0).ready entity=mon.1 client_cookie=dfed9f92d35bcfad server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:14.967 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f0987fff700 1 -- 172.21.15.154:0/1679282799 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f098800d570 con 0x7f0998136690 2022-11-27T17:02:14.967 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f0987fff700 1 -- 172.21.15.154:0/1679282799 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0988000e90 con 0x7f0998136690 2022-11-27T17:02:14.968 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f0987fff700 1 -- 172.21.15.154:0/1679282799 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f098800d8c0 con 0x7f0998136690 2022-11-27T17:02:14.968 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f099f302700 1 -- 172.21.15.154:0/1679282799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0998136690 msgr2=0x7f0998136a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f099f302700 1 --2- 172.21.15.154:0/1679282799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0998136690 0x7f0998136a70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f099809a3b0 tx=0x7f0988006df0).stop 2022-11-27T17:02:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f099f302700 1 -- 172.21.15.154:0/1679282799 shutdown_connections 2022-11-27T17:02:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f099f302700 1 --2- 172.21.15.154:0/1679282799 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f0998136690 0x7f0998136a70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:14.969 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f099f302700 1 --2- 172.21.15.154:0/1679282799 >> v2:172.21.15.154:3300/0 conn(0x7f0998067c60 0x7f099805ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f099f302700 1 --2- 172.21.15.154:0/1679282799 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f099809b540 0x7f099809a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f099f302700 1 -- 172.21.15.154:0/1679282799 >> 172.21.15.154:0/1679282799 conn(0x7f099806c360 msgr2=0x7f0998061670 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f099f302700 1 -- 172.21.15.154:0/1679282799 shutdown_connections 2022-11-27T17:02:14.970 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.955+0000 7f099f302700 1 -- 172.21.15.154:0/1679282799 wait complete. 2022-11-27T17:02:14.971 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.956+0000 7f099f302700 1 Processor -- start 2022-11-27T17:02:14.971 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.956+0000 7f099f302700 1 -- start start 2022-11-27T17:02:14.971 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.956+0000 7f099f302700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0998067c60 0x7f0998146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:14.971 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.956+0000 7f099f302700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099809b540 0x7f0998131d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:14.972 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099f302700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f0998136690 0x7f0998132240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:14.972 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099f302700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09980982c0 con 0x7f0998067c60 2022-11-27T17:02:14.972 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099f302700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09981351e0 con 0x7f099809b540 2022-11-27T17:02:14.972 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099f302700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0998135410 con 0x7f0998136690 2022-11-27T17:02:14.973 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099d09e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0998067c60 0x7f0998146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:14.973 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099d09e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0998067c60 0x7f0998146aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50082/0 (socket says 172.21.15.154:50082) 2022-11-27T17:02:14.973 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099d09e700 1 -- 172.21.15.154:0/1276731265 learned_addr learned my addr 172.21.15.154:0/1276731265 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099d89f700 1 --2- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f0998136690 0x7f0998132240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099c89d700 1 --2- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099809b540 0x7f0998131d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099d09e700 1 -- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f0998136690 msgr2=0x7f0998132240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099d09e700 1 --2- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f0998136690 0x7f0998132240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:14.974 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099d09e700 1 -- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099809b540 msgr2=0x7f0998131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099d09e700 1 --2- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099809b540 0x7f0998131d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099d09e700 1 -- 172.21.15.154:0/1276731265 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099400b1e0 con 0x7f0998067c60 2022-11-27T17:02:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099c89d700 1 --2- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099809b540 0x7f0998131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T17:02:14.975 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099d09e700 1 --2- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0998067c60 0x7f0998146aa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f099809a3b0 tx=0x7f0988014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:14.976 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f09867fc700 1 -- 172.21.15.154:0/1276731265 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0988013e90 con 0x7f0998067c60 2022-11-27T17:02:14.976 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f09867fc700 1 -- 172.21.15.154:0/1276731265 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0988002f70 con 0x7f0998067c60 2022-11-27T17:02:14.976 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f09867fc700 1 -- 172.21.15.154:0/1276731265 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f098801e5f0 con 0x7f0998067c60 2022-11-27T17:02:14.977 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099f302700 1 -- 172.21.15.154:0/1276731265 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098800b180 con 0x7f0998067c60 2022-11-27T17:02:14.977 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.957+0000 7f099f302700 1 -- 172.21.15.154:0/1276731265 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0998147730 con 0x7f0998067c60 2022-11-27T17:02:14.977 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.958+0000 7f09867fc700 1 -- 172.21.15.154:0/1276731265 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f0988025030 con 0x7f0998067c60 2022-11-27T17:02:14.978 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.958+0000 7f099f302700 1 -- 172.21.15.154:0/1276731265 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0998098100 con 0x7f0998067c60 2022-11-27T17:02:14.978 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.958+0000 7f09867fc700 1 -- 172.21.15.154:0/1276731265 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3192+0+0 (secure 0 0 0) 0x7f098804b550 con 0x7f0998067c60 2022-11-27T17:02:14.979 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:14.963+0000 7f09867fc700 1 -- 172.21.15.154:0/1276731265 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f0988013120 con 0x7f0998067c60 2022-11-27T17:02:15.075 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.072+0000 7f099f302700 1 -- 172.21.15.154:0/1276731265 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0998057a60 con 0x7f0998067c60 2022-11-27T17:02:15.826 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.824+0000 7f09867fc700 1 -- 172.21.15.154:0/1276731265 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v17) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0988023020 con 0x7f0998067c60 2022-11-27T17:02:15.828 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.827+0000 7f099f302700 1 -- 172.21.15.154:0/1276731265 >> v1:172.21.15.154:6816/104574 conn(0x7f0978038da0 legacy=0x7f097803b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:15.829 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.827+0000 7f099f302700 1 -- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0998067c60 msgr2=0x7f0998146aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:15.829 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.827+0000 7f099f302700 1 --2- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0998067c60 0x7f0998146aa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f099809a3b0 tx=0x7f0988014000).stop 2022-11-27T17:02:15.829 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.827+0000 7f099f302700 1 -- 172.21.15.154:0/1276731265 shutdown_connections 2022-11-27T17:02:15.829 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.827+0000 7f099f302700 1 --2- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f0998136690 0x7f0998132240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:15.830 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.827+0000 7f099f302700 1 --2- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0998067c60 0x7f0998146aa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:15.830 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.827+0000 7f099f302700 1 --2- 172.21.15.154:0/1276731265 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f099809b540 0x7f0998131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:15.830 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.827+0000 7f099f302700 1 -- 172.21.15.154:0/1276731265 >> 172.21.15.154:0/1276731265 conn(0x7f099806c360 msgr2=0x7f09981331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:15.831 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.827+0000 7f099f302700 1 -- 172.21.15.154:0/1276731265 shutdown_connections 2022-11-27T17:02:15.831 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.828+0000 7f099f302700 1 -- 172.21.15.154:0/1276731265 wait complete. 2022-11-27T17:02:15.843 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2022-11-27T17:02:15.854 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-11-27T17:02:15.971 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.963+0000 7ff8e8a10700 1 Processor -- start 2022-11-27T17:02:15.972 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.963+0000 7ff8e8a10700 1 -- start start 2022-11-27T17:02:15.972 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.964+0000 7ff8e8a10700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff8e405d430 0x7ff8e409aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:15.972 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.964+0000 7ff8e8a10700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 0x7ff8e4140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:15.972 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.964+0000 7ff8e8a10700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e405d810 con 0x7ff8e4099810 2022-11-27T17:02:15.973 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.964+0000 7ff8e8a10700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e409a550 con 0x7ff8e405d430 2022-11-27T17:02:15.973 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.964+0000 7ff8e8a10700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e413a420 con 0x7ff8e413e8c0 2022-11-27T17:02:15.973 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.964+0000 7ff8e259c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff8e405d430 0x7ff8e409aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:15.974 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.964+0000 7ff8e2d9d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 0x7ff8e4140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:15.974 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.964+0000 7ff8e259c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff8e405d430 0x7ff8e409aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50098/0 (socket says 172.21.15.154:50098) 2022-11-27T17:02:15.975 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.964+0000 7ff8e1d9b700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff8e4099810 0x7ff8e4139d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40652/0 (socket says 172.21.15.154:40652) 2022-11-27T17:02:15.975 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.964+0000 7ff8e259c700 1 -- 172.21.15.154:0/1760696023 learned_addr learned my addr 172.21.15.154:0/1760696023 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:15.975 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e2d9d700 1 -- 172.21.15.154:0/1760696023 >> v2:172.21.15.154:3300/0 conn(0x7ff8e405d430 msgr2=0x7ff8e409aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:15.976 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e2d9d700 1 --2- 172.21.15.154:0/1760696023 >> v2:172.21.15.154:3300/0 conn(0x7ff8e405d430 0x7ff8e409aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:15.976 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e2d9d700 1 -- 172.21.15.154:0/1760696023 >> v1:172.21.15.154:6789/0 conn(0x7ff8e4099810 legacy=0x7ff8e4139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:15.976 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e2d9d700 1 -- 172.21.15.154:0/1760696023 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e4139640 con 0x7ff8e413e8c0 2022-11-27T17:02:15.977 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e259c700 1 --2- 172.21.15.154:0/1760696023 >> v2:172.21.15.154:3300/0 conn(0x7ff8e405d430 0x7ff8e409aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:15.977 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e2d9d700 1 --2- 172.21.15.154:0/1760696023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 0x7ff8e4140cb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff8dc0032f0 tx=0x7ff8dc0091d0).ready entity=mon.1 client_cookie=2aa11165be76a6dd server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:15.977 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e159a700 1 -- 172.21.15.154:0/1760696023 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8dc00b6c0 con 0x7ff8e413e8c0 2022-11-27T17:02:15.977 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e159a700 1 -- 172.21.15.154:0/1760696023 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc00b820 con 0x7ff8e413e8c0 2022-11-27T17:02:15.978 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e159a700 1 -- 172.21.15.154:0/1760696023 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8dc00b990 con 0x7ff8e413e8c0 2022-11-27T17:02:15.978 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e8a10700 1 -- 172.21.15.154:0/1760696023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 msgr2=0x7ff8e4140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:15.978 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e8a10700 1 --2- 172.21.15.154:0/1760696023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 0x7ff8e4140cb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff8dc0032f0 tx=0x7ff8dc0091d0).stop 2022-11-27T17:02:15.978 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e8a10700 1 -- 172.21.15.154:0/1760696023 shutdown_connections 2022-11-27T17:02:15.979 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e8a10700 1 --2- 172.21.15.154:0/1760696023 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 0x7ff8e4140cb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:15.979 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e8a10700 1 --2- 172.21.15.154:0/1760696023 >> v2:172.21.15.154:3300/0 conn(0x7ff8e405d430 0x7ff8e409aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:15.979 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e8a10700 1 -- 172.21.15.154:0/1760696023 >> 172.21.15.154:0/1760696023 conn(0x7ff8e40616e0 msgr2=0x7ff8e405cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:15.980 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e8a10700 1 -- 172.21.15.154:0/1760696023 shutdown_connections 2022-11-27T17:02:15.980 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.965+0000 7ff8e8a10700 1 -- 172.21.15.154:0/1760696023 wait complete. 2022-11-27T17:02:15.980 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.966+0000 7ff8e8a10700 1 Processor -- start 2022-11-27T17:02:15.981 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.966+0000 7ff8e8a10700 1 -- start start 2022-11-27T17:02:15.981 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e8a10700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e405d430 0x7ff8e413d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:15.981 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e8a10700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7ff8e4099810 0x7ff8e412e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:15.981 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e8a10700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 0x7ff8e412e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:15.982 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e8a10700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e413da50 con 0x7ff8e405d430 2022-11-27T17:02:15.982 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e8a10700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e413ccc0 con 0x7ff8e413e8c0 2022-11-27T17:02:15.982 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e8a10700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e413bd70 con 0x7ff8e4099810 2022-11-27T17:02:15.982 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e405d430 0x7ff8e413d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:15.983 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e2d9d700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 0x7ff8e412e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:15.983 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e1d9b700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7ff8e4099810 0x7ff8e412e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:15.983 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e259c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e405d430 0x7ff8e413d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50114/0 (socket says 172.21.15.154:50114) 2022-11-27T17:02:15.984 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e259c700 1 -- 172.21.15.154:0/456977436 learned_addr learned my addr 172.21.15.154:0/456977436 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:15.984 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e259c700 1 -- 172.21.15.154:0/456977436 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7ff8e4099810 msgr2=0x7ff8e412e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:15.984 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e259c700 1 --2- 172.21.15.154:0/456977436 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7ff8e4099810 0x7ff8e412e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:15.985 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e259c700 1 -- 172.21.15.154:0/456977436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 msgr2=0x7ff8e412e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:15.985 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e259c700 1 --2- 172.21.15.154:0/456977436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 0x7ff8e412e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:15.985 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e259c700 1 -- 172.21.15.154:0/456977436 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d800b1e0 con 0x7ff8e405d430 2022-11-27T17:02:15.985 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8e259c700 1 --2- 172.21.15.154:0/456977436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e405d430 0x7ff8e413d110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff8e413ce00 tx=0x7ff8d8006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:15.986 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8d37fe700 1 -- 172.21.15.154:0/456977436 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8d8012600 con 0x7ff8e405d430 2022-11-27T17:02:15.986 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.967+0000 7ff8d37fe700 1 -- 172.21.15.154:0/456977436 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8012760 con 0x7ff8e405d430 2022-11-27T17:02:15.986 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.968+0000 7ff8e8a10700 1 -- 172.21.15.154:0/456977436 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8dc009470 con 0x7ff8e405d430 2022-11-27T17:02:15.986 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.968+0000 7ff8d37fe700 1 -- 172.21.15.154:0/456977436 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8d8012ac0 con 0x7ff8e405d430 2022-11-27T17:02:15.986 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.968+0000 7ff8e8a10700 1 -- 172.21.15.154:0/456977436 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e4137570 con 0x7ff8e405d430 2022-11-27T17:02:15.987 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.968+0000 7ff8d37fe700 1 -- 172.21.15.154:0/456977436 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7ff8d8022070 con 0x7ff8e405d430 2022-11-27T17:02:15.987 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.968+0000 7ff8d37fe700 1 -- 172.21.15.154:0/456977436 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3574+0+0 (secure 0 0 0) 0x7ff8d804ade0 con 0x7ff8e405d430 2022-11-27T17:02:15.987 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.969+0000 7ff8e8a10700 1 -- 172.21.15.154:0/456977436 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e413cb10 con 0x7ff8e405d430 2022-11-27T17:02:15.988 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:15.972+0000 7ff8d37fe700 1 -- 172.21.15.154:0/456977436 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7ff8d80336d0 con 0x7ff8e405d430 2022-11-27T17:02:16.116 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.114+0000 7ff8e8a10700 1 -- 172.21.15.154:0/456977436 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff8e41398e0 con 0x7ff8e405d430 2022-11-27T17:02:16.122 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.120+0000 7ff8d37fe700 1 -- 172.21.15.154:0/456977436 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff8d80332c0 con 0x7ff8e405d430 2022-11-27T17:02:16.127 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.125+0000 7ff8e8a10700 1 -- 172.21.15.154:0/456977436 >> v1:172.21.15.154:6816/104574 conn(0x7ff8c0038da0 legacy=0x7ff8c003b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.128 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.125+0000 7ff8e8a10700 1 -- 172.21.15.154:0/456977436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e405d430 msgr2=0x7ff8e413d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.128 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.125+0000 7ff8e8a10700 1 --2- 172.21.15.154:0/456977436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e405d430 0x7ff8e413d110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff8e413ce00 tx=0x7ff8d8006eb0).stop 2022-11-27T17:02:16.128 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.125+0000 7ff8e8a10700 1 -- 172.21.15.154:0/456977436 shutdown_connections 2022-11-27T17:02:16.128 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.125+0000 7ff8e8a10700 1 --2- 172.21.15.154:0/456977436 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7ff8e413e8c0 0x7ff8e412e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.129 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.125+0000 7ff8e8a10700 1 --2- 172.21.15.154:0/456977436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8e405d430 0x7ff8e413d110 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.129 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.125+0000 7ff8e8a10700 1 --2- 172.21.15.154:0/456977436 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7ff8e4099810 0x7ff8e412e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.129 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.125+0000 7ff8e8a10700 1 -- 172.21.15.154:0/456977436 >> 172.21.15.154:0/456977436 conn(0x7ff8e40616e0 msgr2=0x7ff8e4133260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:16.130 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.126+0000 7ff8e8a10700 1 -- 172.21.15.154:0/456977436 shutdown_connections 2022-11-27T17:02:16.130 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.126+0000 7ff8e8a10700 1 -- 172.21.15.154:0/456977436 wait complete. 2022-11-27T17:02:16.130 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 3 and data pool 4 2022-11-27T17:02:16.144 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T17:02:16.260 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.257+0000 7f3b03fe1700 1 Processor -- start 2022-11-27T17:02:16.260 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.258+0000 7f3b03fe1700 1 -- start start 2022-11-27T17:02:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.258+0000 7f3b03fe1700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3afc139aa0 0x7f3afc139e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.269 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.258+0000 7f3b03fe1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc141810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.270 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.258+0000 7f3b03fe1700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3afc0a7430 con 0x7f3afc05cb90 2022-11-27T17:02:16.270 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.258+0000 7f3b03fe1700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3afc05c690 con 0x7f3afc139aa0 2022-11-27T17:02:16.270 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.258+0000 7f3b03fe1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc13a370 con 0x7f3afc060490 2022-11-27T17:02:16.270 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.258+0000 7f3b0257e700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.271 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.258+0000 7f3b01d7d700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3afc139aa0 0x7f3afc139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.271 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b0157c700 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3afc05cb90 0x7f3afc05fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40656/0 (socket says 172.21.15.154:40656) 2022-11-27T17:02:16.271 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b0157c700 1 -- 172.21.15.154:0/1067712277 learned_addr learned my addr 172.21.15.154:0/1067712277 (peer_addr_for_me v1:172.21.15.154:0/0) 2022-11-27T17:02:16.271 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b0257e700 1 -- 172.21.15.154:0/1067712277 >> v2:172.21.15.154:3300/0 conn(0x7f3afc139aa0 msgr2=0x7f3afc139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:16.271 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b0257e700 1 --2- 172.21.15.154:0/1067712277 >> v2:172.21.15.154:3300/0 conn(0x7f3afc139aa0 0x7f3afc139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.272 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b0257e700 1 -- 172.21.15.154:0/1067712277 >> v1:172.21.15.154:6789/0 conn(0x7f3afc05cb90 legacy=0x7f3afc05fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.272 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b0257e700 1 -- 172.21.15.154:0/1067712277 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc05d350 con 0x7f3afc060490 2022-11-27T17:02:16.272 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b0257e700 1 --2- 172.21.15.154:0/1067712277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc141810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f3af40032f0 tx=0x7f3af40091d0).ready entity=mon.1 client_cookie=187114cdee55c845 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:16.273 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b00d7b700 1 -- 172.21.15.154:0/1067712277 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af400b6c0 con 0x7f3afc060490 2022-11-27T17:02:16.273 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b00d7b700 1 -- 172.21.15.154:0/1067712277 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af400b820 con 0x7f3afc060490 2022-11-27T17:02:16.273 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b00d7b700 1 -- 172.21.15.154:0/1067712277 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af400b990 con 0x7f3afc060490 2022-11-27T17:02:16.273 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b03fe1700 1 -- 172.21.15.154:0/1067712277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 msgr2=0x7f3afc141810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.274 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.259+0000 7f3b03fe1700 1 --2- 172.21.15.154:0/1067712277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc141810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f3af40032f0 tx=0x7f3af40091d0).stop 2022-11-27T17:02:16.274 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.260+0000 7f3b03fe1700 1 -- 172.21.15.154:0/1067712277 shutdown_connections 2022-11-27T17:02:16.274 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.260+0000 7f3b03fe1700 1 --2- 172.21.15.154:0/1067712277 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc141810 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.274 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.260+0000 7f3b03fe1700 1 --2- 172.21.15.154:0/1067712277 >> v2:172.21.15.154:3300/0 conn(0x7f3afc139aa0 0x7f3afc139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.274 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.260+0000 7f3b03fe1700 1 -- 172.21.15.154:0/1067712277 >> 172.21.15.154:0/1067712277 conn(0x7f3afc0671b0 msgr2=0x7f3afc05dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:16.275 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.260+0000 7f3b03fe1700 1 -- 172.21.15.154:0/1067712277 shutdown_connections 2022-11-27T17:02:16.275 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.260+0000 7f3b03fe1700 1 -- 172.21.15.154:0/1067712277 wait complete. 2022-11-27T17:02:16.275 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.260+0000 7f3b03fe1700 1 Processor -- start 2022-11-27T17:02:16.275 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.260+0000 7f3b03fe1700 1 -- start start 2022-11-27T17:02:16.276 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b03fe1700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3afc05cb90 0x7f3afc1149e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.276 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b03fe1700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.276 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b03fe1700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc139aa0 0x7f3afc145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.276 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b03fe1700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc1153b0 con 0x7f3afc139aa0 2022-11-27T17:02:16.277 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b03fe1700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc145850 con 0x7f3afc060490 2022-11-27T17:02:16.277 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b03fe1700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc1158a0 con 0x7f3afc05cb90 2022-11-27T17:02:16.277 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0157c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.277 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0257e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc139aa0 0x7f3afc145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.277 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0157c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc145a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49468/0 (socket says 172.21.15.154:49468) 2022-11-27T17:02:16.278 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0157c700 1 -- 172.21.15.154:0/2687164279 learned_addr learned my addr 172.21.15.154:0/2687164279 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:16.279 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b01d7d700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3afc05cb90 0x7f3afc1149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.279 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0257e700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc139aa0 0x7f3afc145f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50126/0 (socket says 172.21.15.154:50126) 2022-11-27T17:02:16.279 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0157c700 1 -- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3afc05cb90 msgr2=0x7f3afc1149e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.280 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0157c700 1 --2- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3afc05cb90 0x7f3afc1149e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.280 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0157c700 1 -- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc139aa0 msgr2=0x7f3afc145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.281 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0157c700 1 --2- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc139aa0 0x7f3afc145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.281 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0157c700 1 -- 172.21.15.154:0/2687164279 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af4009470 con 0x7f3afc060490 2022-11-27T17:02:16.281 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0257e700 1 --2- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc139aa0 0x7f3afc145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:16.281 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.261+0000 7f3b0157c700 1 --2- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc145a40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3afc060f20 tx=0x7f3af8007c40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:16.282 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.262+0000 7f3aeaffd700 1 -- 172.21.15.154:0/2687164279 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af80070d0 con 0x7f3afc060490 2022-11-27T17:02:16.282 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.262+0000 7f3aeaffd700 1 -- 172.21.15.154:0/2687164279 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af8007230 con 0x7f3afc060490 2022-11-27T17:02:16.282 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.262+0000 7f3b03fe1700 1 -- 172.21.15.154:0/2687164279 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3afc0a5c40 con 0x7f3afc060490 2022-11-27T17:02:16.282 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.262+0000 7f3aeaffd700 1 -- 172.21.15.154:0/2687164279 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af8005610 con 0x7f3afc060490 2022-11-27T17:02:16.282 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.262+0000 7f3b03fe1700 1 -- 172.21.15.154:0/2687164279 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc0a62c0 con 0x7f3afc060490 2022-11-27T17:02:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.263+0000 7f3aeaffd700 1 -- 172.21.15.154:0/2687164279 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f3af8010070 con 0x7f3afc060490 2022-11-27T17:02:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.263+0000 7f3aeaffd700 1 -- 172.21.15.154:0/2687164279 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3674+0+0 (secure 0 0 0) 0x7f3af8049de0 con 0x7f3afc060490 2022-11-27T17:02:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.263+0000 7f3b03fe1700 1 -- 172.21.15.154:0/2687164279 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3afc1167b0 con 0x7f3afc060490 2022-11-27T17:02:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.266+0000 7f3aeaffd700 1 -- 172.21.15.154:0/2687164279 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f3af802ece0 con 0x7f3afc060490 2022-11-27T17:02:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.378+0000 7f3b03fe1700 1 -- 172.21.15.154:0/2687164279 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3afc114830 con 0x7f3afc060490 2022-11-27T17:02:16.381 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.378+0000 7f3aeaffd700 1 -- 172.21.15.154:0/2687164279 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+10613 (secure 0 0 0) 0x7f3af8032300 con 0x7f3afc060490 2022-11-27T17:02:16.381 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:02:16.382 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":18,"fsid":"5adefcc8-72cd-433b-bd69-ec1eac0a6bf2","created":"2022-11-27T17:01:59.370943+0000","modified":"2022-11-27T17:02:16.114627+0000","last_up_change":"2022-11-27T17:02:05.307461+0000","last_in_change":"2022-11-27T17:02:05.307461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-11-27T17:02:01.324848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T17:02:08.341335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-11-27T17:02:13.979882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-11-27T17:02:15.104497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3061801a-e670-4a13-9a7d-6c526f88ef28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6808","nonce":105154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6809","nonce":105154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6811","nonce":105154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6810","nonce":105154}]},"public_addr":"172.21.15.154:6808/105154","cluster_addr":"172.21.15.154:6809/105154","heartbeat_back_addr":"172.21.15.154:6811/105154","heartbeat_front_addr":"172.21.15.154:6810/105154","state":["exists","up"]},{"osd":1,"uuid":"6906d403-7e5e-4ac7-9739-9178ecf56f2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6800","nonce":105155}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6801","nonce":105155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6803","nonce":105155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6802","nonce":105155}]},"public_addr":"172.21.15.154:6800/105155","cluster_addr":"172.21.15.154:6801/105155","heartbeat_back_addr":"172.21.15.154:6803/105155","heartbeat_front_addr":"172.21.15.154:6802/105155","state":["exists","up"]},{"osd":2,"uuid":"5bcff406-28bc-465d-8c76-56626ab7936a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6804","nonce":105156}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6805","nonce":105156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6807","nonce":105156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6806","nonce":105156}]},"public_addr":"172.21.15.154:6804/105156","cluster_addr":"172.21.15.154:6805/105156","heartbeat_back_addr":"172.21.15.154:6807/105156","heartbeat_front_addr":"172.21.15.154:6806/105156","state":["exists","up"]},{"osd":3,"uuid":"d6f7e6fc-d825-4e61-b433-6d5cf4075863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6812","nonce":105157}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6813","nonce":105157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6815","nonce":105157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6814","nonce":105157}]},"public_addr":"172.21.15.154:6812/105157","cluster_addr":"172.21.15.154:6813/105157","heartbeat_back_addr":"172.21.15.154:6815/105157","heartbeat_front_addr":"172.21.15.154:6814/105157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.393188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.290316+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.346843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.384664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T17:02:16.384 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.380+0000 7f3b03fe1700 1 -- 172.21.15.154:0/2687164279 >> v1:172.21.15.154:6816/104574 conn(0x7f3adc038e30 legacy=0x7f3adc03b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.384 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.380+0000 7f3b03fe1700 1 -- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 msgr2=0x7f3afc145a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.380+0000 7f3b03fe1700 1 --2- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc145a40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3afc060f20 tx=0x7f3af8007c40).stop 2022-11-27T17:02:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.380+0000 7f3b03fe1700 1 -- 172.21.15.154:0/2687164279 shutdown_connections 2022-11-27T17:02:16.386 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.380+0000 7f3b03fe1700 1 --2- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3afc139aa0 0x7f3afc145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.386 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.380+0000 7f3b03fe1700 1 --2- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f3afc05cb90 0x7f3afc1149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.387 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.380+0000 7f3b03fe1700 1 --2- 172.21.15.154:0/2687164279 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f3afc060490 0x7f3afc145a40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.387 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.380+0000 7f3b03fe1700 1 -- 172.21.15.154:0/2687164279 >> 172.21.15.154:0/2687164279 conn(0x7f3afc0671b0 msgr2=0x7f3afc0aca60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.380+0000 7f3b03fe1700 1 -- 172.21.15.154:0/2687164279 shutdown_connections 2022-11-27T17:02:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.380+0000 7f3b03fe1700 1 -- 172.21.15.154:0/2687164279 wait complete. 2022-11-27T17:02:16.392 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T17:02:16.496 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.494+0000 7fc962e8c700 1 Processor -- start 2022-11-27T17:02:16.496 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.494+0000 7fc962e8c700 1 -- start start 2022-11-27T17:02:16.502 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.494+0000 7fc962e8c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c0923b0 0x7fc95c092790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.503 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.494+0000 7fc962e8c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc95c132c60 0x7fc95c133370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.503 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.494+0000 7fc962e8c700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95c091d50 con 0x7fc95c133860 2022-11-27T17:02:16.505 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.494+0000 7fc962e8c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95c094660 con 0x7fc95c132c60 2022-11-27T17:02:16.505 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.494+0000 7fc962e8c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c091f80 con 0x7fc95c0923b0 2022-11-27T17:02:16.505 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.494+0000 7fc960c28700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c0923b0 0x7fc95c092790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc95bfff700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc95c132c60 0x7fc95c133370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc960c28700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c0923b0 0x7fc95c092790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49478/0 (socket says 172.21.15.154:49478) 2022-11-27T17:02:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc960c28700 1 -- 172.21.15.154:0/3136550016 learned_addr learned my addr 172.21.15.154:0/3136550016 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc960c28700 1 -- 172.21.15.154:0/3136550016 >> v2:172.21.15.154:3300/0 conn(0x7fc95c132c60 msgr2=0x7fc95c133370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:16.507 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc960c28700 1 --2- 172.21.15.154:0/3136550016 >> v2:172.21.15.154:3300/0 conn(0x7fc95c132c60 0x7fc95c133370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.507 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc960c28700 1 -- 172.21.15.154:0/3136550016 >> v1:172.21.15.154:6789/0 conn(0x7fc95c133860 legacy=0x7fc95c1361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.508 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc960c28700 1 -- 172.21.15.154:0/3136550016 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c092c80 con 0x7fc95c0923b0 2022-11-27T17:02:16.508 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc95b7fe700 1 -- 172.21.15.154:0/3136550016 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386596087 0 0) 0x7fc95c091d50 con 0x7fc95c133860 2022-11-27T17:02:16.508 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc95bfff700 1 --2- 172.21.15.154:0/3136550016 >> v2:172.21.15.154:3300/0 conn(0x7fc95c132c60 0x7fc95c133370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:16.509 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc960c28700 1 --2- 172.21.15.154:0/3136550016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c0923b0 0x7fc95c092790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc950002f70 tx=0x7fc950006eb0).ready entity=mon.1 client_cookie=e20f74ebc1500897 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:16.509 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc95b7fe700 1 -- 172.21.15.154:0/3136550016 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95000d570 con 0x7fc95c0923b0 2022-11-27T17:02:16.510 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc95b7fe700 1 -- 172.21.15.154:0/3136550016 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950000e90 con 0x7fc95c0923b0 2022-11-27T17:02:16.510 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.495+0000 7fc95b7fe700 1 -- 172.21.15.154:0/3136550016 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95000d8c0 con 0x7fc95c0923b0 2022-11-27T17:02:16.511 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.496+0000 7fc962e8c700 1 -- 172.21.15.154:0/3136550016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c0923b0 msgr2=0x7fc95c092790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.511 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.496+0000 7fc962e8c700 1 --2- 172.21.15.154:0/3136550016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c0923b0 0x7fc95c092790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc950002f70 tx=0x7fc950006eb0).stop 2022-11-27T17:02:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.496+0000 7fc962e8c700 1 -- 172.21.15.154:0/3136550016 shutdown_connections 2022-11-27T17:02:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.496+0000 7fc962e8c700 1 --2- 172.21.15.154:0/3136550016 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c0923b0 0x7fc95c092790 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.513 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.496+0000 7fc962e8c700 1 --2- 172.21.15.154:0/3136550016 >> v2:172.21.15.154:3300/0 conn(0x7fc95c132c60 0x7fc95c133370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.513 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.496+0000 7fc962e8c700 1 -- 172.21.15.154:0/3136550016 >> 172.21.15.154:0/3136550016 conn(0x7fc95c05ba50 msgr2=0x7fc95c05be60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.496+0000 7fc962e8c700 1 -- 172.21.15.154:0/3136550016 shutdown_connections 2022-11-27T17:02:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.496+0000 7fc962e8c700 1 -- 172.21.15.154:0/3136550016 wait complete. 2022-11-27T17:02:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.496+0000 7fc962e8c700 1 Processor -- start 2022-11-27T17:02:16.515 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.496+0000 7fc962e8c700 1 -- start start 2022-11-27T17:02:16.515 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc962e8c700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fc95c0923b0 0x7fc95c144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc962e8c700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95c132c60 0x7fc95c135230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc962e8c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c133860 0x7fc95c0620d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.517 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc962e8c700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c063550 con 0x7fc95c132c60 2022-11-27T17:02:16.517 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc962e8c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c063390 con 0x7fc95c133860 2022-11-27T17:02:16.517 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc962e8c700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c144dc0 con 0x7fc95c0923b0 2022-11-27T17:02:16.518 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc960c28700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fc95c0923b0 0x7fc95c144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.518 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc961429700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c133860 0x7fc95c0620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.519 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc960c28700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fc95c0923b0 0x7fc95c144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3302/0 says I am v2:172.21.15.154:55366/0 (socket says 172.21.15.154:55366) 2022-11-27T17:02:16.519 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc961429700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c133860 0x7fc95c0620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49488/0 (socket says 172.21.15.154:49488) 2022-11-27T17:02:16.520 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc960c28700 1 -- 172.21.15.154:0/362872253 learned_addr learned my addr 172.21.15.154:0/362872253 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:16.520 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.497+0000 7fc95bfff700 1 --2- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95c132c60 0x7fc95c135230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.521 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc961429700 1 -- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fc95c0923b0 msgr2=0x7fc95c144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.521 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc961429700 1 --2- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fc95c0923b0 0x7fc95c144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.522 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc961429700 1 -- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95c132c60 msgr2=0x7fc95c135230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.522 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc961429700 1 --2- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95c132c60 0x7fc95c135230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.522 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc961429700 1 -- 172.21.15.154:0/362872253 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94800b180 con 0x7fc95c133860 2022-11-27T17:02:16.523 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc960c28700 1 --2- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fc95c0923b0 0x7fc95c144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:16.523 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc95bfff700 1 --2- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95c132c60 0x7fc95c135230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:16.524 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc961429700 1 --2- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c133860 0x7fc95c0620d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fc954001250 tx=0x7fc954007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:16.524 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc959ffb700 1 -- 172.21.15.154:0/362872253 <== mon.1 v2:172.21.15.154:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc95400ce30 con 0x7fc95c133860 2022-11-27T17:02:16.525 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc959ffb700 1 -- 172.21.15.154:0/362872253 <== mon.1 v2:172.21.15.154:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc954006aa0 con 0x7fc95c133860 2022-11-27T17:02:16.525 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc962e8c700 1 -- 172.21.15.154:0/362872253 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95000b1e0 con 0x7fc95c133860 2022-11-27T17:02:16.526 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc959ffb700 1 -- 172.21.15.154:0/362872253 <== mon.1 v2:172.21.15.154:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc954006e00 con 0x7fc95c133860 2022-11-27T17:02:16.526 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.498+0000 7fc962e8c700 1 -- 172.21.15.154:0/362872253 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95c063970 con 0x7fc95c133860 2022-11-27T17:02:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.499+0000 7fc959ffb700 1 -- 172.21.15.154:0/362872253 <== mon.1 v2:172.21.15.154:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fc954010030 con 0x7fc95c133860 2022-11-27T17:02:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.499+0000 7fc959ffb700 1 -- 172.21.15.154:0/362872253 <== mon.1 v2:172.21.15.154:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3674+0+0 (secure 0 0 0) 0x7fc95404a630 con 0x7fc95c133860 2022-11-27T17:02:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.499+0000 7fc962e8c700 1 -- 172.21.15.154:0/362872253 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95c063690 con 0x7fc95c133860 2022-11-27T17:02:16.529 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.502+0000 7fc959ffb700 1 -- 172.21.15.154:0/362872253 <== mon.1 v2:172.21.15.154:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fc95c063690 con 0x7fc95c133860 2022-11-27T17:02:16.614 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.613+0000 7fc962e8c700 1 -- 172.21.15.154:0/362872253 --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc95c063690 con 0x7fc95c133860 2022-11-27T17:02:16.616 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.613+0000 7fc959ffb700 1 -- 172.21.15.154:0/362872253 <== mon.1 v2:172.21.15.154:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+10613 (secure 0 0 0) 0x7fc95c063690 con 0x7fc95c133860 2022-11-27T17:02:16.616 INFO:teuthology.orchestra.run.smithi154.stdout: 2022-11-27T17:02:16.616 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":18,"fsid":"5adefcc8-72cd-433b-bd69-ec1eac0a6bf2","created":"2022-11-27T17:01:59.370943+0000","modified":"2022-11-27T17:02:16.114627+0000","last_up_change":"2022-11-27T17:02:05.307461+0000","last_in_change":"2022-11-27T17:02:05.307461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-11-27T17:02:01.324848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T17:02:08.341335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-11-27T17:02:13.979882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-11-27T17:02:15.104497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3061801a-e670-4a13-9a7d-6c526f88ef28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6808","nonce":105154}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6809","nonce":105154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6811","nonce":105154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6810","nonce":105154}]},"public_addr":"172.21.15.154:6808/105154","cluster_addr":"172.21.15.154:6809/105154","heartbeat_back_addr":"172.21.15.154:6811/105154","heartbeat_front_addr":"172.21.15.154:6810/105154","state":["exists","up"]},{"osd":1,"uuid":"6906d403-7e5e-4ac7-9739-9178ecf56f2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6800","nonce":105155}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6801","nonce":105155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6803","nonce":105155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6802","nonce":105155}]},"public_addr":"172.21.15.154:6800/105155","cluster_addr":"172.21.15.154:6801/105155","heartbeat_back_addr":"172.21.15.154:6803/105155","heartbeat_front_addr":"172.21.15.154:6802/105155","state":["exists","up"]},{"osd":2,"uuid":"5bcff406-28bc-465d-8c76-56626ab7936a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6804","nonce":105156}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6805","nonce":105156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6807","nonce":105156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6806","nonce":105156}]},"public_addr":"172.21.15.154:6804/105156","cluster_addr":"172.21.15.154:6805/105156","heartbeat_back_addr":"172.21.15.154:6807/105156","heartbeat_front_addr":"172.21.15.154:6806/105156","state":["exists","up"]},{"osd":3,"uuid":"d6f7e6fc-d825-4e61-b433-6d5cf4075863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6812","nonce":105157}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6813","nonce":105157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6815","nonce":105157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.154:6814","nonce":105157}]},"public_addr":"172.21.15.154:6812/105157","cluster_addr":"172.21.15.154:6813/105157","heartbeat_back_addr":"172.21.15.154:6815/105157","heartbeat_front_addr":"172.21.15.154:6814/105157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.393188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.290316+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.346843+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T17:02:03.384664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T17:02:16.618 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.614+0000 7fc962e8c700 1 -- 172.21.15.154:0/362872253 >> v1:172.21.15.154:6816/104574 conn(0x7fc93c038da0 legacy=0x7fc93c03b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.618 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.614+0000 7fc962e8c700 1 -- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c133860 msgr2=0x7fc95c0620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.618 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.614+0000 7fc962e8c700 1 --2- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c133860 0x7fc95c0620d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fc954001250 tx=0x7fc954007ef0).stop 2022-11-27T17:02:16.618 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.615+0000 7fc962e8c700 1 -- 172.21.15.154:0/362872253 shutdown_connections 2022-11-27T17:02:16.619 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.615+0000 7fc962e8c700 1 --2- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fc95c133860 0x7fc95c0620d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.619 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.615+0000 7fc962e8c700 1 --2- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7fc95c0923b0 0x7fc95c144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.619 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.615+0000 7fc962e8c700 1 --2- 172.21.15.154:0/362872253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc95c132c60 0x7fc95c135230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.619 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.615+0000 7fc962e8c700 1 -- 172.21.15.154:0/362872253 >> 172.21.15.154:0/362872253 conn(0x7fc95c05ba50 msgr2=0x7fc95c093880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:16.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.615+0000 7fc962e8c700 1 -- 172.21.15.154:0/362872253 shutdown_connections 2022-11-27T17:02:16.620 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.615+0000 7fc962e8c700 1 -- 172.21.15.154:0/362872253 wait complete. 2022-11-27T17:02:16.627 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-11-27T17:02:16.732 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.730+0000 7f17cc531700 1 Processor -- start 2022-11-27T17:02:16.733 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.730+0000 7f17cc531700 1 -- start start 2022-11-27T17:02:16.738 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17cc531700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f17c40923b0 0x7f17c4092790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.739 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17cc531700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c4132c60 0x7f17c4133370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.739 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17cc531700 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c4091d50 con 0x7f17c4133860 2022-11-27T17:02:16.739 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17cc531700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c4094660 con 0x7f17c40923b0 2022-11-27T17:02:16.740 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17cc531700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4091f80 con 0x7f17c4132c60 2022-11-27T17:02:16.740 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17c9acc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c4132c60 0x7f17c4133370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.740 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17ca2cd700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f17c40923b0 0x7f17c4092790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.740 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17c9acc700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c4132c60 0x7f17c4133370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49498/0 (socket says 172.21.15.154:49498) 2022-11-27T17:02:16.741 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17c9acc700 1 -- 172.21.15.154:0/1074102708 learned_addr learned my addr 172.21.15.154:0/1074102708 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:16.741 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17ca2cd700 1 -- 172.21.15.154:0/1074102708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c4132c60 msgr2=0x7f17c4133370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T17:02:16.741 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17ca2cd700 1 --2- 172.21.15.154:0/1074102708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c4132c60 0x7f17c4133370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.742 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17ca2cd700 1 -- 172.21.15.154:0/1074102708 >> v1:172.21.15.154:6789/0 conn(0x7f17c4133860 legacy=0x7f17c41361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.742 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17c92cb700 1 -- 172.21.15.154:0/1074102708 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094170575 0 0) 0x7f17c4091d50 con 0x7f17c4133860 2022-11-27T17:02:16.742 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17ca2cd700 1 -- 172.21.15.154:0/1074102708 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c4092c80 con 0x7f17c40923b0 2022-11-27T17:02:16.742 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.731+0000 7f17c9acc700 1 --2- 172.21.15.154:0/1074102708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c4132c60 0x7f17c4133370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:16.743 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17ca2cd700 1 --2- 172.21.15.154:0/1074102708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c40923b0 0x7f17c4092790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f17c0002f70 tx=0x7f17c0006e80).ready entity=mon.0 client_cookie=7d64f747e9bb1bae server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:16.743 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17c92cb700 1 -- 172.21.15.154:0/1074102708 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17c0012070 con 0x7f17c40923b0 2022-11-27T17:02:16.743 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17c92cb700 1 -- 172.21.15.154:0/1074102708 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0000e90 con 0x7f17c40923b0 2022-11-27T17:02:16.743 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17c92cb700 1 -- 172.21.15.154:0/1074102708 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17c000f7f0 con 0x7f17c40923b0 2022-11-27T17:02:16.744 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17cc531700 1 -- 172.21.15.154:0/1074102708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c40923b0 msgr2=0x7f17c4092790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.744 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17cc531700 1 --2- 172.21.15.154:0/1074102708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c40923b0 0x7f17c4092790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f17c0002f70 tx=0x7f17c0006e80).stop 2022-11-27T17:02:16.744 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17cc531700 1 -- 172.21.15.154:0/1074102708 shutdown_connections 2022-11-27T17:02:16.744 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17cc531700 1 --2- 172.21.15.154:0/1074102708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c40923b0 0x7f17c4092790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.745 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17cc531700 1 --2- 172.21.15.154:0/1074102708 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c4132c60 0x7f17c4133370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.745 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17cc531700 1 -- 172.21.15.154:0/1074102708 >> 172.21.15.154:0/1074102708 conn(0x7f17c405ba50 msgr2=0x7f17c405be60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:16.745 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17cc531700 1 -- 172.21.15.154:0/1074102708 shutdown_connections 2022-11-27T17:02:16.745 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.732+0000 7f17cc531700 1 -- 172.21.15.154:0/1074102708 wait complete. 2022-11-27T17:02:16.746 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.733+0000 7f17cc531700 1 Processor -- start 2022-11-27T17:02:16.746 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.733+0000 7f17cc531700 1 -- start start 2022-11-27T17:02:16.746 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17cc531700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c40923b0 0x7f17c4063200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.746 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17cc531700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f17c4132c60 0x7f17c4144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.746 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17cc531700 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c4133860 0x7f17c41412f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:16.747 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17cc531700 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4062840 con 0x7f17c4133860 2022-11-27T17:02:16.747 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17cc531700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4068140 con 0x7f17c40923b0 2022-11-27T17:02:16.747 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17cc531700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17c4144590 con 0x7f17c4132c60 2022-11-27T17:02:16.747 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17ca2cd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c40923b0 0x7f17c4063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.748 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17c9acc700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f17c4132c60 0x7f17c4144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.748 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17ca2cd700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c40923b0 0x7f17c4063200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.154:3301/0 says I am v2:172.21.15.154:49500/0 (socket says 172.21.15.154:49500) 2022-11-27T17:02:16.748 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17ca2cd700 1 -- 172.21.15.154:0/3528712355 learned_addr learned my addr 172.21.15.154:0/3528712355 (peer_addr_for_me v2:172.21.15.154:0/0) 2022-11-27T17:02:16.748 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17caace700 1 --2- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c4133860 0x7f17c41412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T17:02:16.748 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17c9acc700 1 -- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c40923b0 msgr2=0x7f17c4063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.749 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17c9acc700 1 --2- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c40923b0 0x7f17c4063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.749 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17c9acc700 1 -- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c4133860 msgr2=0x7f17c41412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:16.749 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17c9acc700 1 --2- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c4133860 0x7f17c41412f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:16.749 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17c9acc700 1 -- 172.21.15.154:0/3528712355 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b400c2e0 con 0x7f17c4132c60 2022-11-27T17:02:16.750 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17ca2cd700 1 --2- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c40923b0 0x7f17c4063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:16.750 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17caace700 1 --2- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c4133860 0x7f17c41412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T17:02:16.750 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.734+0000 7f17c9acc700 1 --2- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f17c4132c60 0x7f17c4144c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f17b400d000 tx=0x7f17b4008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T17:02:16.750 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.735+0000 7f17b37fe700 1 -- 172.21.15.154:0/3528712355 <== mon.2 v2:172.21.15.154:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17b400dc10 con 0x7f17c4132c60 2022-11-27T17:02:16.751 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.735+0000 7f17b37fe700 1 -- 172.21.15.154:0/3528712355 <== mon.2 v2:172.21.15.154:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b400dd70 con 0x7f17c4132c60 2022-11-27T17:02:16.751 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.735+0000 7f17cc531700 1 -- 172.21.15.154:0/3528712355 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c000d590 con 0x7f17c4132c60 2022-11-27T17:02:16.751 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.735+0000 7f17b37fe700 1 -- 172.21.15.154:0/3528712355 <== mon.2 v2:172.21.15.154:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17b4010630 con 0x7f17c4132c60 2022-11-27T17:02:16.751 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.735+0000 7f17cc531700 1 -- 172.21.15.154:0/3528712355 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c4135bb0 con 0x7f17c4132c60 2022-11-27T17:02:16.752 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.736+0000 7f17b37fe700 1 -- 172.21.15.154:0/3528712355 <== mon.2 v2:172.21.15.154:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f17b4020070 con 0x7f17c4132c60 2022-11-27T17:02:16.752 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.736+0000 7f17cc531700 1 -- 172.21.15.154:0/3528712355 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c40a8ab0 con 0x7f17c4132c60 2022-11-27T17:02:16.752 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.736+0000 7f17b37fe700 1 -- 172.21.15.154:0/3528712355 <== mon.2 v2:172.21.15.154:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3674+0+0 (secure 0 0 0) 0x7f17b404ae50 con 0x7f17c4132c60 2022-11-27T17:02:16.753 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.739+0000 7f17b37fe700 1 -- 172.21.15.154:0/3528712355 <== mon.2 v2:172.21.15.154:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f17b402fba0 con 0x7f17c4132c60 2022-11-27T17:02:16.885 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:16.883+0000 7f17cc531700 1 -- 172.21.15.154:0/3528712355 --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f17c4057a60 con 0x7f17c4132c60 2022-11-27T17:02:17.121 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.119+0000 7f17b37fe700 1 -- 172.21.15.154:0/3528712355 <== mon.2 v2:172.21.15.154:3302/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f17b4009170 con 0x7f17c4132c60 2022-11-27T17:02:17.123 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.121+0000 7f17cc531700 1 -- 172.21.15.154:0/3528712355 >> v1:172.21.15.154:6816/104574 conn(0x7f17a4038da0 legacy=0x7f17a403b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:17.124 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.121+0000 7f17cc531700 1 -- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f17c4132c60 msgr2=0x7f17c4144c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T17:02:17.124 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.121+0000 7f17cc531700 1 --2- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f17c4132c60 0x7f17c4144c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f17b400d000 tx=0x7f17b4008040).stop 2022-11-27T17:02:17.124 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.121+0000 7f17cc531700 1 -- 172.21.15.154:0/3528712355 shutdown_connections 2022-11-27T17:02:17.125 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.121+0000 7f17cc531700 1 --2- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c4133860 0x7f17c41412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:17.125 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.121+0000 7f17cc531700 1 --2- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f17c40923b0 0x7f17c4063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:17.126 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.121+0000 7f17cc531700 1 --2- 172.21.15.154:0/3528712355 >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f17c4132c60 0x7f17c4144c50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T17:02:17.127 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.121+0000 7f17cc531700 1 -- 172.21.15.154:0/3528712355 >> 172.21.15.154:0/3528712355 conn(0x7f17c405ba50 msgr2=0x7f17c405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T17:02:17.127 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.121+0000 7f17cc531700 1 -- 172.21.15.154:0/3528712355 shutdown_connections 2022-11-27T17:02:17.127 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.121+0000 7f17cc531700 1 -- 172.21.15.154:0/3528712355 wait complete. 2022-11-27T17:02:17.137 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 2} 2022-11-27T17:02:17.137 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2022-11-27T17:02:17.254 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.241+0000 7f00eff1c700 1 Processor -- start 2022-11-27T17:02:17.254 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.241+0000 7f00eff1c700 1 -- start start 2022-11-27T17:02:17.254 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.242+0000 7f00eff1c700 1 --2- >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7f00e805d430 0x7f00e809aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:17.255 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.242+0000 7f00eff1c700 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f00e8099810 0x7f00e8139d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:17.255 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.242+0000 7f00eff1c700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f00e813a200 0x7f00e805ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T17:02:17.255 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.242+0000 7f00eff1c700 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e805d810 con 0x7f00e8099810 2022-11-27T17:02:17.255 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.242+0000 7f00eff1c700 1 -- --> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e809a550 con 0x7f00e805d430 2022-11-27T17:02:17.256 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.242+0000 7f00eff1c700 1 -- --> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f00e8139640 con 0x7f00e813a200 2022-11-27T17:02:17.256 INFO:teuthology.orchestra.run.smithi154.stderr:2022-11-27T17:02:17.242+0000 7f00ee4b9700 1 --2- >> [v2:172.21.15.154:3302/0,v1:172.21.15.154:6791/0] conn(0x7f00e813a200 0x7f00e805ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 req