2022-11-27T14:51:28.288 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-27T14:51:28.290 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} --name yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088588 --verbose -- /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088588/orig.config.yaml 2022-11-27T14:51:28.351 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-11-27T14:51:28.428 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088588 branch: main description: fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} email: ceph-qa@ceph.io first_in_suite: false job_id: '7088588' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'install ceph/octopus latest ' name: yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: main ceph: cephfs: max_mds: 2 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on osd down out interval zero: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - mds.a - mds.b - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - client.0 - - client.1 sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 sleep_before_teardown: 0 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 targets: smithi039.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNwrUDKayEO2S7Mie7vNpN3SFJxfD23cC8vb20TEAN5NN7zsq40dzi+Wy3JHohppj7rs7k1UoBsDyY7SFs+Ty19rJahL3b/toR2dUu+5jTzWaAmcTe4qfQXj5nPodGX7K57+oF9+jV8MO2f1yeFt8AV0wOhCV+6xH9SJNKI9ZR/WtkAUjJHFNj4z9/BgxBBCOnFGUuorDm+N42s/sn2AJu1JQm0QVDEfACbcfgsVVeKaM8A91xhP2PBEL9LyHkpPLoIEcaJgLX2T/o9tY/I36lVEWVdneB3qtifL9Feo7J7QCQCVbLYQ4ZazzW6evTZk1ckF7B8GpDMdW6J5FlRrKckfAU/P18uTssiSsqUBwVURd0bYudF+hl0CCIBd79eZnt/2CvEUif+RuYtlpowTwISmQ8QKWV3xe4hpVJiF0/s/RAeYanf4h6g8h8+5LgDjwjwDRhrX/TctT1Lyuw7lQNfJxmPi482Qn7DXNq+gyNnH6Vwgw8r4UvxIi6eVnWYck= smithi102.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuf9IbSu5GEtrGO8nP78s5VBQ6HsMZOC92ssnMH6UAkn+RuwoJmXrwpbAjsXw9D0VGQEQwGljiluO5vFF74FGfYhjjvE2yzlpaj86TxTG846xCEtySGvz566diD4Z+eM3Qg7CSz3FiFY8NNEGkEj3BJ84equmr66PrlqIi+oEcsSE8rnT7XHoJPmsI/ppjMgqZn9FIyhyRXKV0u0NPsKf00k/PZ1ZNeRJCfSn30nimMmoOBoDE0iv+Lh/Mk+Sy4seY7Z51XmtSNrxmMMfkVwkygxDYzDR1WLeF/GvUBYLIGkkDYIWfu8yLGXxHzcNCH03TFYENdNbaSE87bV6nkRbFvrs4d+E4ySaddkKFdpwmkm+Lfkoz/6biykIIuEQRxXlMkcfImkqmpAJeUTkYTPpQ1R0xIx4ZB0ILeQfip0tVZq3FGnScGjWWDYdfmtW8NJ9zzEiNJElZKpqwFKiE8sNADtoWNOT1DKX+rP3JKRlm0eu/Wy8MDBadSLOB6Q8b+lM= smithi133.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4Z11Kkjqy08SJTm/famI+TGALhySgEe4RuwwvwwoRSdcliFfIepI19Zrf3QVEGwPrsmBz08b+8LZFoTE/VM84JHBNWvxGnjattIadpv+80FJNJW3eFu9agc5apKpe2GGECuPZCZSXBgZ7IUhJQhxGySjYZ03KRqye+wi6sp3q13kw+IBWDX95ucyP+KCrnVeTovSbeCFiOuSXg+MKaDHBsI1LGCmiMVIFBHnd029kJTtIU4gtkUBggEBCjuwZbj8a7B9lmOh+zYP/xYc1oV0rZ+EDUI1Lmv4y6cEAi7k8S3aLbu5iWgypKYHNXdONxxLvcNpdLwYa+4JE4zb60zrYh43PqpuprqgD+7XYs+pIpPUS2zPjFW2fzJ+rAJKdOtEXZapk5jL/Ba/DAjUxLtSuvBW7x50tJm9xBPy7slL557INBDZLnQnSAZRfY1ueLW+oNflLJu5uuj5poCZt6KQrKUUky9HjNCiknfkBpSBW52z1UqSsRMAluztFZeI0ngE= tasks: - install: branch: octopus exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-rook - ceph-mgr-cephadm - cephadm - ceph-volume extra_packages: - librados2 - print: '**** done installing octopus' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client octopus - print: '**** done ceph' - ceph-fuse: null - print: '**** done octopus client' - workunit: clients: all: - suites/fsstress.sh - print: '**** done fsstress' - mds_pre_upgrade: null - print: '**** done mds pre-upgrade sequence' - install.upgrade: mon.a: branch: quincy mon.b: branch: quincy - print: '**** done install.upgrade both hosts' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - ceph.stop: - mds.* - ceph.restart: daemons: - mds.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph osd dump -f json-pretty - ceph versions - ceph osd require-osd-release quincy - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph.healthy: null - print: '**** done ceph.restart' - workunit: clients: all: - suites/fsstress.sh - print: '**** done fsstress' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-23_17:24:26 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.11998 2022-11-27T14:51:28.429 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa; will attempt to use it 2022-11-27T14:51:28.430 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa/tasks 2022-11-27T14:51:28.430 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-11-27T14:51:28.432 INFO:teuthology.task.internal:Checking packages... 2022-11-27T14:51:28.456 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '697b0d99d9a52df92b20c8e053dcd46c75f364a9' 2022-11-27T14:51:28.456 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T14:51:28.457 INFO:teuthology.packaging:ref: None 2022-11-27T14:51:28.457 INFO:teuthology.packaging:tag: None 2022-11-27T14:51:28.458 INFO:teuthology.packaging:branch: main 2022-11-27T14:51:28.458 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:51:28.458 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=main 2022-11-27T14:51:28.749 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-982.g2bb1aa26 2022-11-27T14:51:28.749 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-11-27T14:51:28.756 INFO:teuthology.task.internal:no buildpackages task found 2022-11-27T14:51:28.757 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-11-27T14:51:28.777 INFO:teuthology.task.internal:Saving configuration 2022-11-27T14:51:28.799 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-11-27T14:51:28.806 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-27T14:51:28.832 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 14:45:35.279037', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOEV4fbWdBWQOc44thZpgHutw6cBnhPJhebuD0Q1MMQJv5gO9meepanmz3zKh3m+y6m7kOlOGm6VobnqQ33ZSG0='} 2022-11-27T14:51:28.859 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 14:45:35.277873', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBElS1Z5dpuWkw10rY4rrdrsQA8taU18hUEhiT3WjXoYaZJePHxbFPsj6g9zPEqoJ1yIN+k6Gfagq+BU5EzXg5+E='} 2022-11-27T14:51:28.883 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 14:45:35.276388', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFKbxim3g2Ons69c1abk529MjTLG4RRMUF/vXbv08Dw3t3Aolb5vKgMGXsPtPaF73lBPaxPQJQU34LV8j06IyRU='} 2022-11-27T14:51:28.884 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-11-27T14:51:28.890 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-11-27T14:51:28.891 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['client.0'] 2022-11-27T14:51:28.891 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['client.1'] 2022-11-27T14:51:28.891 INFO:teuthology.run_tasks:Running task console_log... 2022-11-27T14:51:29.011 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f39b43f91e0>, signals=[15]) 2022-11-27T14:51:29.012 INFO:teuthology.run_tasks:Running task internal.connect... 2022-11-27T14:51:29.020 INFO:teuthology.task.internal:Opening connections... 2022-11-27T14:51:29.020 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2022-11-27T14:51:29.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T14:51:29.105 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2022-11-27T14:51:29.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T14:51:29.178 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2022-11-27T14:51:29.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T14:51:29.278 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-11-27T14:51:29.285 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2022-11-27T14:51:29.308 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2022-11-27T14:51:29.308 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2022-11-27T14:51:29.368 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2022-11-27T14:51:29.369 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="8" 2022-11-27T14:51:29.369 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2022-11-27T14:51:29.369 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2022-11-27T14:51:29.369 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="8" 2022-11-27T14:51:29.370 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el8" 2022-11-27T14:51:29.370 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 8" 2022-11-27T14:51:29.370 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2022-11-27T14:51:29.371 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-11-27T14:51:29.371 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2022-11-27T14:51:29.371 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T14:51:29.371 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T14:51:29.372 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-11-27T14:51:29.373 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2022-11-27T14:51:29.392 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2022-11-27T14:51:29.411 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2022-11-27T14:51:29.412 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2022-11-27T14:51:29.470 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2022-11-27T14:51:29.470 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="8" 2022-11-27T14:51:29.470 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2022-11-27T14:51:29.471 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2022-11-27T14:51:29.471 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="8" 2022-11-27T14:51:29.471 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el8" 2022-11-27T14:51:29.471 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 8" 2022-11-27T14:51:29.472 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2022-11-27T14:51:29.472 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-11-27T14:51:29.472 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2022-11-27T14:51:29.472 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T14:51:29.473 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T14:51:29.473 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-11-27T14:51:29.473 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2022-11-27T14:51:29.493 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2022-11-27T14:51:29.513 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2022-11-27T14:51:29.514 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2022-11-27T14:51:29.573 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2022-11-27T14:51:29.573 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8" 2022-11-27T14:51:29.573 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2022-11-27T14:51:29.574 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2022-11-27T14:51:29.574 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8" 2022-11-27T14:51:29.574 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2022-11-27T14:51:29.575 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 8" 2022-11-27T14:51:29.575 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2022-11-27T14:51:29.575 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-11-27T14:51:29.575 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2022-11-27T14:51:29.575 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T14:51:29.576 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T14:51:29.576 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-11-27T14:51:29.576 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2022-11-27T14:51:29.594 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-11-27T14:51:29.605 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-11-27T14:51:29.613 INFO:teuthology.task.internal:Checking for old test directory... 2022-11-27T14:51:29.613 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2022-11-27T14:51:29.616 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2022-11-27T14:51:29.619 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2022-11-27T14:51:29.636 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-11-27T14:51:29.643 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-11-27T14:51:29.644 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2022-11-27T14:51:29.678 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2022-11-27T14:51:29.680 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2022-11-27T14:51:29.713 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-11-27T14:51:29.795 INFO:teuthology.run_tasks:Running task kernel... 2022-11-27T14:51:29.814 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-11-27T14:51:29.815 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-11-27T14:51:29.816 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T14:51:29.817 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T14:51:29.817 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T14:51:29.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T14:51:29.836 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2022-11-27T14:51:29.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T14:51:29.837 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2022-11-27T14:51:29.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T14:51:29.839 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2022-11-27T14:51:29.893 INFO:teuthology.orchestra.run.smithi102.stdout:4.18.0-408.el8.x86_64 2022-11-27T14:51:29.894 INFO:teuthology.task.kernel:Running kernel on smithi102: 4.18.0-408.el8.x86_64 2022-11-27T14:51:29.894 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum install -y kernel 2022-11-27T14:51:29.897 INFO:teuthology.orchestra.run.smithi133.stdout:4.18.0-408.el8.x86_64 2022-11-27T14:51:29.898 INFO:teuthology.task.kernel:Running kernel on smithi133: 4.18.0-408.el8.x86_64 2022-11-27T14:51:29.898 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum install -y kernel 2022-11-27T14:51:29.900 INFO:teuthology.orchestra.run.smithi039.stdout:4.18.0-408.el8.x86_64 2022-11-27T14:51:29.900 INFO:teuthology.task.kernel:Running kernel on smithi039: 4.18.0-408.el8.x86_64 2022-11-27T14:51:29.900 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum install -y kernel 2022-11-27T14:51:33.149 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-8 - AppStream 27 MB/s | 26 MB 00:00 2022-11-27T14:51:33.227 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-8 - AppStream 26 MB/s | 26 MB 00:00 2022-11-27T14:51:33.369 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-8 - AppStream 23 MB/s | 26 MB 00:01 2022-11-27T14:51:38.710 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-8 - BaseOS 37 MB/s | 26 MB 00:00 2022-11-27T14:51:38.778 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-8 - BaseOS 28 MB/s | 26 MB 00:00 2022-11-27T14:51:39.016 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-8 - BaseOS 28 MB/s | 26 MB 00:00 2022-11-27T14:51:42.717 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-8 - Extras 171 kB/s | 18 kB 00:00 2022-11-27T14:51:42.760 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-8 - Extras 158 kB/s | 18 kB 00:00 2022-11-27T14:51:43.086 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-8 - PowerTools 21 MB/s | 5.3 MB 00:00 2022-11-27T14:51:43.145 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-8 - PowerTools 20 MB/s | 5.3 MB 00:00 2022-11-27T14:51:43.215 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-8 - Extras 166 kB/s | 18 kB 00:00 2022-11-27T14:51:43.592 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-8 - PowerTools 21 MB/s | 5.3 MB 00:00 2022-11-27T14:51:46.969 INFO:teuthology.orchestra.run.smithi102.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2022-11-27T14:51:47.074 INFO:teuthology.orchestra.run.smithi039.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2022-11-27T14:51:47.993 INFO:teuthology.orchestra.run.smithi133.stdout:Copr repo for python3-asyncssh owned by ceph 914 B/s | 3.5 kB 00:03 2022-11-27T14:51:48.128 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:01 2022-11-27T14:51:48.500 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 10 MB/s | 13 MB 00:01 2022-11-27T14:51:49.085 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2022-11-27T14:51:51.311 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 438 kB/s | 24 kB 00:00 2022-11-27T14:51:51.842 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 381 kB/s | 24 kB 00:00 2022-11-27T14:51:52.268 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 427 kB/s | 24 kB 00:00 2022-11-27T14:51:54.284 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2022-11-27T14:51:54.285 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2022-11-27T14:51:54.424 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T14:51:54.425 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T14:51:54.425 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T14:51:54.570 DEBUG:teuthology.orchestra.run.smithi102:> echo no | sudo yum reinstall kernel || true 2022-11-27T14:51:54.998 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2022-11-27T14:51:54.998 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2022-11-27T14:51:55.140 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T14:51:55.141 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T14:51:55.141 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T14:51:55.149 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:04 ago on Sun 27 Nov 2022 02:51:51 PM UTC. 2022-11-27T14:51:55.268 DEBUG:teuthology.orchestra.run.smithi039:> echo no | sudo yum reinstall kernel || true 2022-11-27T14:51:55.282 INFO:teuthology.orchestra.run.smithi133.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2022-11-27T14:51:55.283 INFO:teuthology.orchestra.run.smithi133.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2022-11-27T14:51:55.418 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T14:51:55.419 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T14:51:55.419 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T14:51:55.553 DEBUG:teuthology.orchestra.run.smithi133:> echo no | sudo yum reinstall kernel || true 2022-11-27T14:51:55.883 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:04 ago on Sun 27 Nov 2022 02:51:51 PM UTC. 2022-11-27T14:51:56.174 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:00:04 ago on Sun 27 Nov 2022 02:51:52 PM UTC. 2022-11-27T14:51:56.227 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T14:51:56.229 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T14:51:56.229 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2022-11-27T14:51:56.229 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T14:51:56.229 INFO:teuthology.orchestra.run.smithi102.stdout:Reinstalling: 2022-11-27T14:51:56.230 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T14:51:56.230 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T14:51:56.230 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T14:51:56.230 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T14:51:56.230 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T14:51:56.231 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 8.5 M 2022-11-27T14:51:56.231 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 0 2022-11-27T14:51:56.231 INFO:teuthology.orchestra.run.smithi102.stderr:Operation aborted. 2022-11-27T14:51:56.296 INFO:teuthology.orchestra.run.smithi102.stdout:Is this ok [y/N]: 2022-11-27T14:51:56.296 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum reinstall -y kernel || true 2022-11-27T14:51:56.877 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:05 ago on Sun 27 Nov 2022 02:51:51 PM UTC. 2022-11-27T14:51:57.054 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T14:51:57.055 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T14:51:57.055 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2022-11-27T14:51:57.055 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T14:51:57.056 INFO:teuthology.orchestra.run.smithi039.stdout:Reinstalling: 2022-11-27T14:51:57.056 INFO:teuthology.orchestra.run.smithi039.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T14:51:57.056 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T14:51:57.056 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T14:51:57.057 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T14:51:57.057 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T14:51:57.058 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 8.5 M 2022-11-27T14:51:57.058 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 0 2022-11-27T14:51:57.059 INFO:teuthology.orchestra.run.smithi039.stderr:Operation aborted. 2022-11-27T14:51:57.127 INFO:teuthology.orchestra.run.smithi039.stdout:Is this ok [y/N]: 2022-11-27T14:51:57.127 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum reinstall -y kernel || true 2022-11-27T14:51:57.283 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T14:51:57.285 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T14:51:57.286 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2022-11-27T14:51:57.286 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T14:51:57.287 INFO:teuthology.orchestra.run.smithi133.stdout:Reinstalling: 2022-11-27T14:51:57.287 INFO:teuthology.orchestra.run.smithi133.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T14:51:57.288 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T14:51:57.288 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T14:51:57.288 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T14:51:57.288 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T14:51:57.289 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 8.5 M 2022-11-27T14:51:57.289 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 0 2022-11-27T14:51:57.289 INFO:teuthology.orchestra.run.smithi133.stderr:Operation aborted. 2022-11-27T14:51:57.347 INFO:teuthology.orchestra.run.smithi133.stdout:Is this ok [y/N]: 2022-11-27T14:51:57.348 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum reinstall -y kernel || true 2022-11-27T14:51:57.738 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:06 ago on Sun 27 Nov 2022 02:51:51 PM UTC. 2022-11-27T14:51:57.928 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:00:05 ago on Sun 27 Nov 2022 02:51:52 PM UTC. 2022-11-27T14:51:57.951 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T14:51:57.953 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T14:51:57.953 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2022-11-27T14:51:57.954 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T14:51:57.954 INFO:teuthology.orchestra.run.smithi102.stdout:Reinstalling: 2022-11-27T14:51:57.955 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T14:51:57.955 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T14:51:57.955 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T14:51:57.956 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T14:51:57.956 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T14:51:57.956 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 8.5 M 2022-11-27T14:51:57.956 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 0 2022-11-27T14:51:57.957 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T14:51:58.338 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-4.18.0-408.el8.x86_64.rpm 22 MB/s | 8.5 MB 00:00 2022-11-27T14:51:58.339 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T14:51:58.340 INFO:teuthology.orchestra.run.smithi102.stdout:Total 22 MB/s | 8.5 MB 00:00 2022-11-27T14:51:58.432 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T14:51:58.832 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T14:51:58.832 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T14:51:58.893 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T14:51:58.893 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T14:51:58.893 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2022-11-27T14:51:58.894 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T14:51:58.894 INFO:teuthology.orchestra.run.smithi039.stdout:Reinstalling: 2022-11-27T14:51:58.894 INFO:teuthology.orchestra.run.smithi039.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T14:51:58.894 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T14:51:58.895 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T14:51:58.895 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T14:51:58.895 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T14:51:58.895 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 8.5 M 2022-11-27T14:51:58.896 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 0 2022-11-27T14:51:58.896 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T14:51:58.990 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T14:51:59.008 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T14:51:59.012 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T14:51:59.012 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T14:51:59.013 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2022-11-27T14:51:59.013 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T14:51:59.013 INFO:teuthology.orchestra.run.smithi133.stdout:Reinstalling: 2022-11-27T14:51:59.013 INFO:teuthology.orchestra.run.smithi133.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T14:51:59.014 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T14:51:59.014 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T14:51:59.014 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T14:51:59.014 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T14:51:59.015 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 8.5 M 2022-11-27T14:51:59.015 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 0 2022-11-27T14:51:59.016 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T14:51:59.126 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-4.18.0-408.el8.x86_64.rpm 37 MB/s | 8.5 MB 00:00 2022-11-27T14:51:59.127 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T14:51:59.127 INFO:teuthology.orchestra.run.smithi039.stdout:Total 37 MB/s | 8.5 MB 00:00 2022-11-27T14:51:59.228 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T14:51:59.244 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-4.18.0-408.el8.x86_64.rpm 37 MB/s | 8.5 MB 00:00 2022-11-27T14:51:59.245 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T14:51:59.246 INFO:teuthology.orchestra.run.smithi133.stdout:Total 37 MB/s | 8.5 MB 00:00 2022-11-27T14:51:59.288 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T14:51:59.340 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T14:51:59.483 INFO:teuthology.orchestra.run.smithi102.stdout: Reinstalling : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T14:51:59.484 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T14:51:59.660 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T14:51:59.660 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T14:51:59.770 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T14:51:59.770 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T14:51:59.839 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T14:51:59.861 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T14:51:59.868 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T14:51:59.868 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T14:51:59.940 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T14:51:59.948 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T14:52:00.150 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T14:52:00.230 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T14:52:00.361 INFO:teuthology.orchestra.run.smithi039.stdout: Reinstalling : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T14:52:00.362 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T14:52:00.390 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T14:52:00.391 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T14:52:00.392 INFO:teuthology.orchestra.run.smithi102.stdout:Reinstalled: 2022-11-27T14:52:00.392 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-4.18.0-408.el8.x86_64 2022-11-27T14:52:00.392 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T14:52:00.393 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T14:52:00.415 INFO:teuthology.orchestra.run.smithi133.stdout: Reinstalling : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T14:52:00.415 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T14:52:00.536 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel --last | head -n 1 2022-11-27T14:52:00.672 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-4.18.0-408.el8.x86_64 Sun 27 Nov 2022 02:51:59 PM UTC 2022-11-27T14:52:00.674 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-408.el8.x86_64 2022-11-27T14:52:00.674 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T14:52:00.674 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2022-11-27T14:52:00.675 DEBUG:teuthology.orchestra.run.smithi102:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-27T14:52:00.713 INFO:teuthology.orchestra.run.smithi102.stdout:ttyS1 2022-11-27T14:52:00.732 DEBUG:teuthology.parallel:result is None 2022-11-27T14:52:00.774 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T14:52:00.775 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T14:52:00.786 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T14:52:00.786 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T14:52:01.293 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T14:52:01.294 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T14:52:01.294 INFO:teuthology.orchestra.run.smithi133.stdout:Reinstalled: 2022-11-27T14:52:01.294 INFO:teuthology.orchestra.run.smithi133.stdout: kernel-4.18.0-408.el8.x86_64 2022-11-27T14:52:01.295 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T14:52:01.295 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T14:52:01.355 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T14:52:01.356 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T14:52:01.356 INFO:teuthology.orchestra.run.smithi039.stdout:Reinstalled: 2022-11-27T14:52:01.356 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-4.18.0-408.el8.x86_64 2022-11-27T14:52:01.356 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T14:52:01.357 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T14:52:01.402 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q kernel --last | head -n 1 2022-11-27T14:52:01.489 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q kernel --last | head -n 1 2022-11-27T14:52:01.535 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-4.18.0-408.el8.x86_64 Sun 27 Nov 2022 02:52:00 PM UTC 2022-11-27T14:52:01.537 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-408.el8.x86_64 2022-11-27T14:52:01.537 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T14:52:01.537 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2022-11-27T14:52:01.538 DEBUG:teuthology.orchestra.run.smithi133:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-27T14:52:01.568 INFO:teuthology.orchestra.run.smithi133.stdout:ttyS1 2022-11-27T14:52:01.584 DEBUG:teuthology.parallel:result is None 2022-11-27T14:52:01.627 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-4.18.0-408.el8.x86_64 Sun 27 Nov 2022 02:52:00 PM UTC 2022-11-27T14:52:01.629 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-408.el8.x86_64 2022-11-27T14:52:01.629 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T14:52:01.630 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-11-27T14:52:01.630 DEBUG:teuthology.orchestra.run.smithi039:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-27T14:52:01.662 INFO:teuthology.orchestra.run.smithi039.stdout:ttyS1 2022-11-27T14:52:01.675 DEBUG:teuthology.parallel:result is None 2022-11-27T14:52:01.675 INFO:teuthology.run_tasks:Running task internal.base... 2022-11-27T14:52:01.685 INFO:teuthology.task.internal:Creating test directory... 2022-11-27T14:52:01.686 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T14:52:01.717 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T14:52:01.720 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T14:52:01.738 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-11-27T14:52:01.748 INFO:teuthology.run_tasks:Running task internal.archive... 2022-11-27T14:52:01.763 INFO:teuthology.task.internal:Creating archive directory... 2022-11-27T14:52:01.764 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T14:52:01.775 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T14:52:01.779 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T14:52:01.826 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-11-27T14:52:01.836 INFO:teuthology.task.internal:Enabling coredump saving... 2022-11-27T14:52:01.836 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T14:52:01.840 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T14:52:01.849 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T14:52:01.869 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T14:52:01.879 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T14:52:01.886 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T14:52:01.896 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T14:52:01.907 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T14:52:01.917 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T14:52:01.918 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-11-27T14:52:01.937 INFO:teuthology.task.internal:Configuring sudo... 2022-11-27T14:52:01.938 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T14:52:01.940 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T14:52:01.943 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T14:52:01.971 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-11-27T14:52:01.986 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-11-27T14:52:01.987 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T14:52:02.008 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T14:52:02.011 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T14:52:02.029 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T14:52:02.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T14:52:02.164 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T14:52:02.250 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T14:52:02.292 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T14:52:02.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T14:52:02.360 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T14:52:02.408 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T14:52:02.463 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T14:52:02.550 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T14:52:02.593 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2022-11-27T14:52:02.593 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T14:52:02.659 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T14:52:02.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T14:52:02.779 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T14:52:02.834 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T14:52:02.866 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2022-11-27T14:52:02.867 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T14:52:02.934 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2022-11-27T14:52:02.938 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2022-11-27T14:52:02.940 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2022-11-27T14:52:02.996 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T14:52:02.997 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T14:52:03.036 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T14:52:03.355 INFO:teuthology.run_tasks:Running task internal.timer... 2022-11-27T14:52:03.368 INFO:teuthology.task.internal:Starting timer... 2022-11-27T14:52:03.369 INFO:teuthology.run_tasks:Running task pcp... 2022-11-27T14:52:03.385 INFO:teuthology.run_tasks:Running task selinux... 2022-11-27T14:52:03.400 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2022-11-27T14:52:03.401 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2022-11-27T14:52:03.453 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2022-11-27T14:52:03.455 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2022-11-27T14:52:03.535 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2022-11-27T14:52:03.536 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2022-11-27T14:52:03.612 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2022-11-27T14:52:03.615 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-11-27T14:52:03.615 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2022-11-27T14:52:03.636 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2022-11-27T14:52:03.637 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2022-11-27T14:52:03.663 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2022-11-27T14:52:03.664 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2022-11-27T14:52:03.688 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2022-11-27T14:52:03.688 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi102.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi133.front.sepia.ceph.com': 'permissive'} 2022-11-27T14:52:03.689 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T14:52:03.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T14:52:03.720 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T14:52:03.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T14:52:03.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T14:52:03.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T14:52:03.777 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-11-27T14:52:03.777 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2022-11-27T14:52:03.807 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2022-11-27T14:52:03.840 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2022-11-27T14:52:03.887 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-11-27T14:52:03.897 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2022-11-27T14:52:03.905 INFO:teuthology.repo_utils:Fetching main from origin 2022-11-27T14:52:04.024 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-11-27T14:52:04.047 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-11-27T14:52:04.050 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi102.front.sepia.ceph.com,smithi133.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-11-27T14:59:03.177 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi102.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com')] 2022-11-27T14:59:03.180 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2022-11-27T14:59:03.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T14:59:03.265 DEBUG:teuthology.orchestra.run.smithi039:> true 2022-11-27T14:59:03.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2022-11-27T14:59:03.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T14:59:03.420 DEBUG:teuthology.orchestra.run.smithi102:> true 2022-11-27T14:59:03.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2022-11-27T14:59:03.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T14:59:03.573 DEBUG:teuthology.orchestra.run.smithi133:> true 2022-11-27T14:59:03.598 INFO:teuthology.run_tasks:Running task clock... 2022-11-27T14:59:03.611 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-11-27T14:59:03.612 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T14:59:03.615 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T14:59:03.618 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T14:59:03.619 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T14:59:03.622 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T14:59:03.623 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T14:59:03.651 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T14:59:03.653 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T14:59:03.664 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T14:59:03.668 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T14:59:03.669 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T14:59:03.683 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T14:59:03.730 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2022-11-27T14:59:03.731 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2022-11-27T14:59:03.741 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2022-11-27T14:59:03.743 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2022-11-27T14:59:03.743 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2022-11-27T14:59:03.754 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2022-11-27T14:59:03.755 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T14:59:03.758 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T14:59:03.768 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T14:59:03.768 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T14:59:03.772 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T14:59:03.781 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T14:59:03.831 INFO:teuthology.orchestra.run.smithi102.stderr:bash: ntpq: command not found 2022-11-27T14:59:03.835 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T14:59:03.835 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2022-11-27T14:59:03.835 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.835 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.836 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.836 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.843 INFO:teuthology.orchestra.run.smithi039.stderr:bash: ntpq: command not found 2022-11-27T14:59:03.844 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2022-11-27T14:59:03.846 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T14:59:03.846 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2022-11-27T14:59:03.847 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.847 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.847 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.847 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.849 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T14:59:03.849 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2022-11-27T14:59:03.849 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.850 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.850 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.850 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:59:03.851 INFO:teuthology.run_tasks:Running task install... 2022-11-27T14:59:03.859 DEBUG:teuthology.task.install:project ceph 2022-11-27T14:59:03.860 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}} 2022-11-27T14:59:03.860 DEBUG:teuthology.task.install:config {'branch': 'octopus', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'} 2022-11-27T14:59:03.860 INFO:teuthology.task.install:Using flavor: default 2022-11-27T14:59:03.867 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-11-27T14:59:03.868 INFO:teuthology.task.install:extra packages: ['librados2'] 2022-11-27T14:59:03.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo lsb_release -is 2022-11-27T14:59:03.926 INFO:teuthology.orchestra.run.smithi039.stdout:CentOSStream 2022-11-27T14:59:03.927 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T14:59:03.928 DEBUG:teuthology.orchestra.run.smithi102:> sudo lsb_release -is 2022-11-27T14:59:03.929 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'octopus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T14:59:03.929 DEBUG:teuthology.orchestra.run.smithi039:> sudo lsb_release -is 2022-11-27T14:59:03.965 INFO:teuthology.orchestra.run.smithi102.stdout:CentOSStream 2022-11-27T14:59:03.967 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T14:59:03.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo lsb_release -is 2022-11-27T14:59:03.968 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'octopus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T14:59:03.968 DEBUG:teuthology.orchestra.run.smithi102:> sudo lsb_release -is 2022-11-27T14:59:04.004 INFO:teuthology.orchestra.run.smithi133.stdout:CentOSStream 2022-11-27T14:59:04.005 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T14:59:04.006 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'octopus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T14:59:04.006 DEBUG:teuthology.orchestra.run.smithi133:> sudo lsb_release -is 2022-11-27T14:59:04.008 INFO:teuthology.orchestra.run.smithi039.stdout:CentOSStream 2022-11-27T14:59:04.009 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T14:59:04.009 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T14:59:04.009 INFO:teuthology.packaging:ref: None 2022-11-27T14:59:04.010 INFO:teuthology.packaging:tag: None 2022-11-27T14:59:04.010 INFO:teuthology.packaging:branch: octopus 2022-11-27T14:59:04.010 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:59:04.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T14:59:04.048 INFO:teuthology.orchestra.run.smithi102.stdout:CentOSStream 2022-11-27T14:59:04.049 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T14:59:04.049 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T14:59:04.050 INFO:teuthology.packaging:ref: None 2022-11-27T14:59:04.050 INFO:teuthology.packaging:tag: None 2022-11-27T14:59:04.050 INFO:teuthology.packaging:branch: octopus 2022-11-27T14:59:04.050 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:59:04.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T14:59:04.085 INFO:teuthology.orchestra.run.smithi133.stdout:CentOSStream 2022-11-27T14:59:04.085 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T14:59:04.085 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T14:59:04.086 INFO:teuthology.packaging:ref: None 2022-11-27T14:59:04.086 INFO:teuthology.packaging:tag: None 2022-11-27T14:59:04.086 INFO:teuthology.packaging:branch: octopus 2022-11-27T14:59:04.087 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:59:04.087 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T14:59:04.230 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/ 2022-11-27T14:59:04.231 INFO:teuthology.task.install.rpm:Package version is 15.2.17-0 2022-11-27T14:59:04.269 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/ 2022-11-27T14:59:04.269 INFO:teuthology.task.install.rpm:Package version is 15.2.17-0 2022-11-27T14:59:04.298 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/ 2022-11-27T14:59:04.299 INFO:teuthology.task.install.rpm:Package version is 15.2.17-0 2022-11-27T14:59:04.304 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T14:59:04.304 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T14:59:04.304 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T14:59:04.317 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T14:59:04.317 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2022-11-27T14:59:04.318 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T14:59:04.334 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T14:59:04.334 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2022-11-27T14:59:04.335 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T14:59:04.339 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2022-11-27T14:59:04.339 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T14:59:04.340 INFO:teuthology.packaging:ref: None 2022-11-27T14:59:04.340 INFO:teuthology.packaging:tag: None 2022-11-27T14:59:04.340 INFO:teuthology.packaging:branch: octopus 2022-11-27T14:59:04.341 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:59:04.341 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/octopus/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T14:59:04.345 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2022-11-27T14:59:04.345 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T14:59:04.346 INFO:teuthology.packaging:ref: None 2022-11-27T14:59:04.346 INFO:teuthology.packaging:tag: None 2022-11-27T14:59:04.346 INFO:teuthology.packaging:branch: octopus 2022-11-27T14:59:04.346 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:59:04.347 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/octopus/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T14:59:04.362 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2022-11-27T14:59:04.362 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T14:59:04.363 INFO:teuthology.packaging:ref: None 2022-11-27T14:59:04.363 INFO:teuthology.packaging:tag: None 2022-11-27T14:59:04.363 INFO:teuthology.packaging:branch: octopus 2022-11-27T14:59:04.363 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:59:04.364 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/octopus/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T14:59:04.408 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T14:59:04.413 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T14:59:04.430 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T14:59:04.486 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T14:59:04.490 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T14:59:04.507 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T14:59:04.553 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2022-11-27T14:59:04.554 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2022-11-27T14:59:04.556 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2022-11-27T14:59:04.557 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2022-11-27T14:59:04.573 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2022-11-27T14:59:04.575 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2022-11-27T14:59:04.815 INFO:teuthology.orchestra.run.smithi102.stdout:64 files removed 2022-11-27T14:59:04.823 INFO:teuthology.orchestra.run.smithi039.stdout:64 files removed 2022-11-27T14:59:04.833 INFO:teuthology.orchestra.run.smithi133.stdout:64 files removed 2022-11-27T14:59:04.844 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw 2022-11-27T14:59:04.855 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw 2022-11-27T14:59:04.861 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw 2022-11-27T14:59:05.277 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 952 kB/s | 68 kB 00:00 2022-11-27T14:59:05.294 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 952 kB/s | 68 kB 00:00 2022-11-27T14:59:05.313 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 898 kB/s | 68 kB 00:00 2022-11-27T14:59:05.387 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 197 kB/s | 15 kB 00:00 2022-11-27T14:59:05.388 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 253 kB/s | 15 kB 00:00 2022-11-27T14:59:05.431 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 183 kB/s | 15 kB 00:00 2022-11-27T14:59:05.470 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 29 kB/s | 1.7 kB 00:00 2022-11-27T14:59:05.478 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 26 kB/s | 1.7 kB 00:00 2022-11-27T14:59:05.531 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 23 kB/s | 1.7 kB 00:00 2022-11-27T14:59:06.198 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-8 - AppStream 37 MB/s | 26 MB 00:00 2022-11-27T14:59:06.306 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-8 - AppStream 32 MB/s | 26 MB 00:00 2022-11-27T14:59:06.858 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-8 - AppStream 20 MB/s | 26 MB 00:01 2022-11-27T14:59:11.531 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-8 - BaseOS 39 MB/s | 26 MB 00:00 2022-11-27T14:59:11.698 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-8 - BaseOS 35 MB/s | 26 MB 00:00 2022-11-27T14:59:12.559 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-8 - BaseOS 42 MB/s | 26 MB 00:00 2022-11-27T14:59:15.539 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-8 - Extras 146 kB/s | 18 kB 00:00 2022-11-27T14:59:15.738 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-8 - Extras 166 kB/s | 18 kB 00:00 2022-11-27T14:59:15.908 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS-8 - PowerTools 21 MB/s | 5.3 MB 00:00 2022-11-27T14:59:16.107 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-8 - PowerTools 21 MB/s | 5.3 MB 00:00 2022-11-27T14:59:16.866 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-8 - Extras 132 kB/s | 18 kB 00:00 2022-11-27T14:59:17.244 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS-8 - PowerTools 21 MB/s | 5.3 MB 00:00 2022-11-27T14:59:19.781 INFO:teuthology.orchestra.run.smithi133.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2022-11-27T14:59:19.790 INFO:teuthology.orchestra.run.smithi102.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2022-11-27T14:59:20.923 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:01 2022-11-27T14:59:21.015 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 12 MB/s | 13 MB 00:01 2022-11-27T14:59:21.984 INFO:teuthology.orchestra.run.smithi039.stdout:Copr repo for python3-asyncssh owned by ceph 970 B/s | 3.5 kB 00:03 2022-11-27T14:59:23.076 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2022-11-27T14:59:24.123 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 396 kB/s | 24 kB 00:00 2022-11-27T14:59:24.232 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 623 kB/s | 24 kB 00:00 2022-11-27T14:59:26.443 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 408 kB/s | 24 kB 00:00 2022-11-27T14:59:27.287 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T14:59:27.289 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T14:59:27.289 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2022-11-27T14:59:27.290 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T14:59:27.290 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2022-11-27T14:59:27.290 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:15.2.17-0.el8 ceph 9.9 M 2022-11-27T14:59:27.290 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2022-11-27T14:59:27.290 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:15.2.17-0.el8 ceph 3.5 M 2022-11-27T14:59:27.291 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:15.2.17-0.el8 ceph 1.9 M 2022-11-27T14:59:27.291 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2022-11-27T14:59:27.291 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:15.2.17-0.el8 ceph 5.6 M 2022-11-27T14:59:27.291 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:15.2.17-0.el8 ceph 21 M 2022-11-27T14:59:27.292 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:15.2.17-0.el8 ceph 25 k 2022-11-27T14:59:27.292 INFO:teuthology.orchestra.run.smithi133.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2022-11-27T14:59:27.292 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T14:59:27.292 INFO:teuthology.orchestra.run.smithi133.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2022-11-27T14:59:27.293 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:15.2.17-0.el8 ceph 508 k 2022-11-27T14:59:27.293 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T14:59:27.293 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2022-11-27T14:59:27.293 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:15.2.17-0.el8 ceph 380 k 2022-11-27T14:59:27.294 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2022-11-27T14:59:27.294 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:15.2.17-0.el8 ceph 3.2 M 2022-11-27T14:59:27.294 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T14:59:27.294 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2022-11-27T14:59:27.295 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:15.2.17-0.el8 ceph 40 k 2022-11-27T14:59:27.295 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:15.2.17-0.el8 ceph 77 k 2022-11-27T14:59:27.295 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:15.2.17-0.el8 ceph 164 k 2022-11-27T14:59:27.295 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2022-11-27T14:59:27.296 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:15.2.17-0.el8 ceph 364 k 2022-11-27T14:59:27.296 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:15.2.17-0.el8 ceph 349 k 2022-11-27T14:59:27.296 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:15.2.17-0.el8 ceph 113 k 2022-11-27T14:59:27.296 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T14:59:27.297 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T14:59:27.297 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T14:59:27.297 INFO:teuthology.orchestra.run.smithi133.stdout:Install 22 Packages 2022-11-27T14:59:27.297 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 2 Packages 2022-11-27T14:59:27.298 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T14:59:27.298 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 48 M 2022-11-27T14:59:27.299 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T14:59:27.432 INFO:teuthology.orchestra.run.smithi133.stdout:(1/24): ceph-base-15.2.17-0.el8.x86_64.rpm 40 MB/s | 5.6 MB 00:00 2022-11-27T14:59:27.457 INFO:teuthology.orchestra.run.smithi133.stdout:(2/24): ceph-selinux-15.2.17-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2022-11-27T14:59:27.472 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T14:59:27.475 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T14:59:27.475 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2022-11-27T14:59:27.475 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T14:59:27.476 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2022-11-27T14:59:27.476 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:15.2.17-0.el8 ceph 9.9 M 2022-11-27T14:59:27.476 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2022-11-27T14:59:27.476 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:15.2.17-0.el8 ceph 3.5 M 2022-11-27T14:59:27.477 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:15.2.17-0.el8 ceph 1.9 M 2022-11-27T14:59:27.477 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2022-11-27T14:59:27.477 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:15.2.17-0.el8 ceph 5.6 M 2022-11-27T14:59:27.477 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:15.2.17-0.el8 ceph 21 M 2022-11-27T14:59:27.478 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:15.2.17-0.el8 ceph 25 k 2022-11-27T14:59:27.478 INFO:teuthology.orchestra.run.smithi102.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2022-11-27T14:59:27.478 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T14:59:27.478 INFO:teuthology.orchestra.run.smithi102.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2022-11-27T14:59:27.478 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:15.2.17-0.el8 ceph 508 k 2022-11-27T14:59:27.479 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T14:59:27.479 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2022-11-27T14:59:27.479 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:15.2.17-0.el8 ceph 380 k 2022-11-27T14:59:27.479 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2022-11-27T14:59:27.480 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:15.2.17-0.el8 ceph 3.2 M 2022-11-27T14:59:27.480 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T14:59:27.480 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2022-11-27T14:59:27.480 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:15.2.17-0.el8 ceph 40 k 2022-11-27T14:59:27.480 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:15.2.17-0.el8 ceph 77 k 2022-11-27T14:59:27.481 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:15.2.17-0.el8 ceph 164 k 2022-11-27T14:59:27.481 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2022-11-27T14:59:27.481 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:15.2.17-0.el8 ceph 364 k 2022-11-27T14:59:27.481 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:15.2.17-0.el8 ceph 349 k 2022-11-27T14:59:27.482 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:15.2.17-0.el8 ceph 113 k 2022-11-27T14:59:27.482 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T14:59:27.482 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T14:59:27.482 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T14:59:27.482 INFO:teuthology.orchestra.run.smithi102.stdout:Install 22 Packages 2022-11-27T14:59:27.483 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 2 Packages 2022-11-27T14:59:27.483 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T14:59:27.484 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 48 M 2022-11-27T14:59:27.484 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T14:59:27.491 INFO:teuthology.orchestra.run.smithi133.stdout:(3/24): libcephfs2-15.2.17-0.el8.x86_64.rpm 15 MB/s | 508 kB 00:00 2022-11-27T14:59:27.525 INFO:teuthology.orchestra.run.smithi133.stdout:(4/24): libradosstriper1-15.2.17-0.el8.x86_64.r 11 MB/s | 380 kB 00:00 2022-11-27T14:59:27.608 INFO:teuthology.orchestra.run.smithi133.stdout:(5/24): librgw2-15.2.17-0.el8.x86_64.rpm 39 MB/s | 3.2 MB 00:00 2022-11-27T14:59:27.617 INFO:teuthology.orchestra.run.smithi102.stdout:(1/24): ceph-base-15.2.17-0.el8.x86_64.rpm 41 MB/s | 5.6 MB 00:00 2022-11-27T14:59:27.659 INFO:teuthology.orchestra.run.smithi133.stdout:(6/24): python3-ceph-argparse-15.2.17-0.el8.x86 799 kB/s | 40 kB 00:00 2022-11-27T14:59:27.667 INFO:teuthology.orchestra.run.smithi102.stdout:(2/24): ceph-selinux-15.2.17-0.el8.x86_64.rpm 511 kB/s | 25 kB 00:00 2022-11-27T14:59:27.676 INFO:teuthology.orchestra.run.smithi133.stdout:(7/24): python3-ceph-common-15.2.17-0.el8.x86_6 4.6 MB/s | 77 kB 00:00 2022-11-27T14:59:27.701 INFO:teuthology.orchestra.run.smithi102.stdout:(3/24): libcephfs2-15.2.17-0.el8.x86_64.rpm 15 MB/s | 508 kB 00:00 2022-11-27T14:59:27.759 INFO:teuthology.orchestra.run.smithi102.stdout:(4/24): libradosstriper1-15.2.17-0.el8.x86_64.r 6.4 MB/s | 380 kB 00:00 2022-11-27T14:59:27.776 INFO:teuthology.orchestra.run.smithi133.stdout:(8/24): ceph-radosgw-15.2.17-0.el8.x86_64.rpm 21 MB/s | 9.9 MB 00:00 2022-11-27T14:59:27.792 INFO:teuthology.orchestra.run.smithi133.stdout:(9/24): python3-cephfs-15.2.17-0.el8.x86_64.rpm 1.4 MB/s | 164 kB 00:00 2022-11-27T14:59:27.818 INFO:teuthology.orchestra.run.smithi133.stdout:(10/24): python3-rados-15.2.17-0.el8.x86_64.rpm 8.5 MB/s | 364 kB 00:00 2022-11-27T14:59:27.843 INFO:teuthology.orchestra.run.smithi133.stdout:(11/24): python3-rbd-15.2.17-0.el8.x86_64.rpm 6.8 MB/s | 349 kB 00:00 2022-11-27T14:59:27.869 INFO:teuthology.orchestra.run.smithi133.stdout:(12/24): python3-rgw-15.2.17-0.el8.x86_64.rpm 2.2 MB/s | 113 kB 00:00 2022-11-27T14:59:27.876 INFO:teuthology.orchestra.run.smithi102.stdout:(5/24): ceph-radosgw-15.2.17-0.el8.x86_64.rpm 25 MB/s | 9.9 MB 00:00 2022-11-27T14:59:27.901 INFO:teuthology.orchestra.run.smithi102.stdout:(6/24): python3-ceph-argparse-15.2.17-0.el8.x86 1.6 MB/s | 40 kB 00:00 2022-11-27T14:59:27.919 INFO:teuthology.orchestra.run.smithi102.stdout:(7/24): python3-ceph-common-15.2.17-0.el8.x86_6 4.5 MB/s | 77 kB 00:00 2022-11-27T14:59:27.920 INFO:teuthology.orchestra.run.smithi133.stdout:(13/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2022-11-27T14:59:27.944 INFO:teuthology.orchestra.run.smithi102.stdout:(8/24): python3-cephfs-15.2.17-0.el8.x86_64.rpm 6.4 MB/s | 164 kB 00:00 2022-11-27T14:59:27.965 INFO:teuthology.orchestra.run.smithi133.stdout:(14/24): python3-prettytable-0.7.2-14.el8.noarc 963 kB/s | 44 kB 00:00 2022-11-27T14:59:27.969 INFO:teuthology.orchestra.run.smithi102.stdout:(9/24): python3-rados-15.2.17-0.el8.x86_64.rpm 14 MB/s | 364 kB 00:00 2022-11-27T14:59:27.994 INFO:teuthology.orchestra.run.smithi133.stdout:(15/24): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2022-11-27T14:59:28.011 INFO:teuthology.orchestra.run.smithi133.stdout:(16/24): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.0 MB/s | 47 kB 00:00 2022-11-27T14:59:28.027 INFO:teuthology.orchestra.run.smithi102.stdout:(10/24): librgw2-15.2.17-0.el8.x86_64.rpm 12 MB/s | 3.2 MB 00:00 2022-11-27T14:59:28.053 INFO:teuthology.orchestra.run.smithi102.stdout:(11/24): python3-rbd-15.2.17-0.el8.x86_64.rpm 4.1 MB/s | 349 kB 00:00 2022-11-27T14:59:28.078 INFO:teuthology.orchestra.run.smithi102.stdout:(12/24): python3-rgw-15.2.17-0.el8.x86_64.rpm 2.2 MB/s | 113 kB 00:00 2022-11-27T14:59:28.129 INFO:teuthology.orchestra.run.smithi102.stdout:(13/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2022-11-27T14:59:28.146 INFO:teuthology.orchestra.run.smithi102.stdout:(14/24): python3-prettytable-0.7.2-14.el8.noarc 2.7 MB/s | 44 kB 00:00 2022-11-27T14:59:28.163 INFO:teuthology.orchestra.run.smithi102.stdout:(15/24): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.7 MB/s | 47 kB 00:00 2022-11-27T14:59:28.196 INFO:teuthology.orchestra.run.smithi102.stdout:(16/24): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2022-11-27T14:59:28.220 INFO:teuthology.orchestra.run.smithi133.stdout:(17/24): ceph-common-15.2.17-0.el8.x86_64.rpm 22 MB/s | 21 MB 00:00 2022-11-27T14:59:28.370 INFO:teuthology.orchestra.run.smithi133.stdout:(18/24): fmt-6.2.1-1.el8.x86_64.rpm 310 kB/s | 116 kB 00:00 2022-11-27T14:59:28.395 INFO:teuthology.orchestra.run.smithi133.stdout:(19/24): gperftools-libs-2.7-9.el8.x86_64.rpm 798 kB/s | 306 kB 00:00 2022-11-27T14:59:28.421 INFO:teuthology.orchestra.run.smithi133.stdout:(20/24): leveldb-1.22-1.el8.x86_64.rpm 903 kB/s | 181 kB 00:00 2022-11-27T14:59:28.437 INFO:teuthology.orchestra.run.smithi133.stdout:(21/24): liboath-2.6.2-3.el8.x86_64.rpm 877 kB/s | 59 kB 00:00 2022-11-27T14:59:28.438 INFO:teuthology.orchestra.run.smithi102.stdout:(17/24): ceph-common-15.2.17-0.el8.x86_64.rpm 22 MB/s | 21 MB 00:00 2022-11-27T14:59:28.455 INFO:teuthology.orchestra.run.smithi133.stdout:(22/24): libunwind-1.3.1-3.el8.x86_64.rpm 1.2 MB/s | 75 kB 00:00 2022-11-27T14:59:28.463 INFO:teuthology.orchestra.run.smithi102.stdout:(18/24): fmt-6.2.1-1.el8.x86_64.rpm 387 kB/s | 116 kB 00:00 2022-11-27T14:59:28.505 INFO:teuthology.orchestra.run.smithi133.stdout:(23/24): librbd1-15.2.17-0.el8.x86_64.rpm 29 MB/s | 1.9 MB 00:00 2022-11-27T14:59:28.514 INFO:teuthology.orchestra.run.smithi102.stdout:(19/24): liboath-2.6.2-3.el8.x86_64.rpm 1.2 MB/s | 59 kB 00:00 2022-11-27T14:59:28.556 INFO:teuthology.orchestra.run.smithi102.stdout:(20/24): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2022-11-27T14:59:28.563 INFO:teuthology.orchestra.run.smithi133.stdout:(24/24): librados2-15.2.17-0.el8.x86_64.rpm 25 MB/s | 3.5 MB 00:00 2022-11-27T14:59:28.566 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T14:59:28.567 INFO:teuthology.orchestra.run.smithi133.stdout:Total 38 MB/s | 48 MB 00:01 2022-11-27T14:59:28.603 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T14:59:28.631 INFO:teuthology.orchestra.run.smithi102.stdout:(21/24): librados2-15.2.17-0.el8.x86_64.rpm 47 MB/s | 3.5 MB 00:00 2022-11-27T14:59:28.647 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T14:59:28.647 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T14:59:28.656 INFO:teuthology.orchestra.run.smithi102.stdout:(22/24): leveldb-1.22-1.el8.x86_64.rpm 832 kB/s | 181 kB 00:00 2022-11-27T14:59:28.682 INFO:teuthology.orchestra.run.smithi102.stdout:(23/24): gperftools-libs-2.7-9.el8.x86_64.rpm 631 kB/s | 306 kB 00:00 2022-11-27T14:59:28.723 INFO:teuthology.orchestra.run.smithi102.stdout:(24/24): librbd1-15.2.17-0.el8.x86_64.rpm 21 MB/s | 1.9 MB 00:00 2022-11-27T14:59:28.727 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T14:59:28.728 INFO:teuthology.orchestra.run.smithi102.stdout:Total 39 MB/s | 48 MB 00:01 2022-11-27T14:59:28.767 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T14:59:28.811 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T14:59:28.812 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T14:59:28.936 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T14:59:28.939 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T14:59:29.103 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T14:59:29.106 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T14:59:29.831 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T14:59:29.834 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T14:59:29.834 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2022-11-27T14:59:29.835 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T14:59:29.835 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2022-11-27T14:59:29.836 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:15.2.17-0.el8 ceph 9.9 M 2022-11-27T14:59:29.836 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2022-11-27T14:59:29.836 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:15.2.17-0.el8 ceph 3.5 M 2022-11-27T14:59:29.837 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:15.2.17-0.el8 ceph 1.9 M 2022-11-27T14:59:29.837 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2022-11-27T14:59:29.838 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:15.2.17-0.el8 ceph 5.6 M 2022-11-27T14:59:29.838 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:15.2.17-0.el8 ceph 21 M 2022-11-27T14:59:29.838 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:15.2.17-0.el8 ceph 25 k 2022-11-27T14:59:29.839 INFO:teuthology.orchestra.run.smithi039.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2022-11-27T14:59:29.839 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T14:59:29.840 INFO:teuthology.orchestra.run.smithi039.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2022-11-27T14:59:29.840 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:15.2.17-0.el8 ceph 508 k 2022-11-27T14:59:29.841 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T14:59:29.841 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2022-11-27T14:59:29.841 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:15.2.17-0.el8 ceph 380 k 2022-11-27T14:59:29.841 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2022-11-27T14:59:29.842 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:15.2.17-0.el8 ceph 3.2 M 2022-11-27T14:59:29.842 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T14:59:29.842 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2022-11-27T14:59:29.843 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:15.2.17-0.el8 ceph 40 k 2022-11-27T14:59:29.843 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:15.2.17-0.el8 ceph 77 k 2022-11-27T14:59:29.843 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:15.2.17-0.el8 ceph 164 k 2022-11-27T14:59:29.843 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2022-11-27T14:59:29.844 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:15.2.17-0.el8 ceph 364 k 2022-11-27T14:59:29.844 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:15.2.17-0.el8 ceph 349 k 2022-11-27T14:59:29.845 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:15.2.17-0.el8 ceph 113 k 2022-11-27T14:59:29.845 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T14:59:29.845 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T14:59:29.845 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T14:59:29.845 INFO:teuthology.orchestra.run.smithi039.stdout:Install 22 Packages 2022-11-27T14:59:29.846 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 2 Packages 2022-11-27T14:59:29.846 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T14:59:29.847 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 48 M 2022-11-27T14:59:29.847 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T14:59:29.920 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T14:59:29.977 INFO:teuthology.orchestra.run.smithi039.stdout:(1/24): ceph-base-15.2.17-0.el8.x86_64.rpm 40 MB/s | 5.6 MB 00:00 2022-11-27T14:59:30.002 INFO:teuthology.orchestra.run.smithi039.stdout:(2/24): ceph-selinux-15.2.17-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2022-11-27T14:59:30.036 INFO:teuthology.orchestra.run.smithi039.stdout:(3/24): libcephfs2-15.2.17-0.el8.x86_64.rpm 15 MB/s | 508 kB 00:00 2022-11-27T14:59:30.069 INFO:teuthology.orchestra.run.smithi039.stdout:(4/24): libradosstriper1-15.2.17-0.el8.x86_64.r 11 MB/s | 380 kB 00:00 2022-11-27T14:59:30.079 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2022-11-27T14:59:30.082 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T14:59:30.145 INFO:teuthology.orchestra.run.smithi039.stdout:(5/24): librgw2-15.2.17-0.el8.x86_64.rpm 43 MB/s | 3.2 MB 00:00 2022-11-27T14:59:30.162 INFO:teuthology.orchestra.run.smithi039.stdout:(6/24): python3-ceph-argparse-15.2.17-0.el8.x86 2.4 MB/s | 40 kB 00:00 2022-11-27T14:59:30.179 INFO:teuthology.orchestra.run.smithi039.stdout:(7/24): python3-ceph-common-15.2.17-0.el8.x86_6 4.5 MB/s | 77 kB 00:00 2022-11-27T14:59:30.216 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2022-11-27T14:59:30.235 INFO:teuthology.orchestra.run.smithi039.stdout:(8/24): python3-cephfs-15.2.17-0.el8.x86_64.rpm 2.9 MB/s | 164 kB 00:00 2022-11-27T14:59:30.271 INFO:teuthology.orchestra.run.smithi039.stdout:(9/24): python3-rados-15.2.17-0.el8.x86_64.rpm 10 MB/s | 364 kB 00:00 2022-11-27T14:59:30.296 INFO:teuthology.orchestra.run.smithi039.stdout:(10/24): python3-rbd-15.2.17-0.el8.x86_64.rpm 14 MB/s | 349 kB 00:00 2022-11-27T14:59:30.308 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2022-11-27T14:59:30.322 INFO:teuthology.orchestra.run.smithi039.stdout:(11/24): python3-rgw-15.2.17-0.el8.x86_64.rpm 4.4 MB/s | 113 kB 00:00 2022-11-27T14:59:30.431 INFO:teuthology.orchestra.run.smithi039.stdout:(12/24): ceph-radosgw-15.2.17-0.el8.x86_64.rpm 17 MB/s | 9.9 MB 00:00 2022-11-27T14:59:30.474 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2022-11-27T14:59:30.480 INFO:teuthology.orchestra.run.smithi039.stdout:(13/24): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2022-11-27T14:59:30.498 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T14:59:30.506 INFO:teuthology.orchestra.run.smithi039.stdout:(14/24): python3-prettytable-0.7.2-14.el8.noarc 1.7 MB/s | 44 kB 00:00 2022-11-27T14:59:30.531 INFO:teuthology.orchestra.run.smithi039.stdout:(15/24): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2022-11-27T14:59:30.557 INFO:teuthology.orchestra.run.smithi039.stdout:(16/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.1 MB/s | 259 kB 00:00 2022-11-27T14:59:30.685 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T14:59:30.766 INFO:teuthology.orchestra.run.smithi039.stdout:(17/24): ceph-common-15.2.17-0.el8.x86_64.rpm 22 MB/s | 21 MB 00:00 2022-11-27T14:59:30.782 INFO:teuthology.orchestra.run.smithi039.stdout:(18/24): fmt-6.2.1-1.el8.x86_64.rpm 465 kB/s | 116 kB 00:00 2022-11-27T14:59:30.832 INFO:teuthology.orchestra.run.smithi039.stdout:(19/24): liboath-2.6.2-3.el8.x86_64.rpm 1.1 MB/s | 59 kB 00:00 2022-11-27T14:59:30.869 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T14:59:30.882 INFO:teuthology.orchestra.run.smithi039.stdout:(20/24): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2022-11-27T14:59:30.958 INFO:teuthology.orchestra.run.smithi039.stdout:(21/24): librados2-15.2.17-0.el8.x86_64.rpm 47 MB/s | 3.5 MB 00:00 2022-11-27T14:59:30.983 INFO:teuthology.orchestra.run.smithi039.stdout:(22/24): gperftools-libs-2.7-9.el8.x86_64.rpm 719 kB/s | 306 kB 00:00 2022-11-27T14:59:31.008 INFO:teuthology.orchestra.run.smithi039.stdout:(23/24): leveldb-1.22-1.el8.x86_64.rpm 745 kB/s | 181 kB 00:00 2022-11-27T14:59:31.041 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T14:59:31.048 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T14:59:31.058 INFO:teuthology.orchestra.run.smithi039.stdout:(24/24): librbd1-15.2.17-0.el8.x86_64.rpm 19 MB/s | 1.9 MB 00:00 2022-11-27T14:59:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T14:59:31.062 INFO:teuthology.orchestra.run.smithi039.stdout:Total 39 MB/s | 48 MB 00:01 2022-11-27T14:59:31.100 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T14:59:31.145 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T14:59:31.148 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T14:59:31.149 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T14:59:31.168 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T14:59:31.246 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T14:59:31.388 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T14:59:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T14:59:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T14:59:31.491 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T14:59:31.631 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T14:59:31.701 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T14:59:31.851 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T14:59:31.911 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T14:59:32.073 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T14:59:32.118 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T14:59:32.126 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T14:59:32.171 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T14:59:32.313 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/26 2022-11-27T14:59:32.358 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/26 2022-11-27T14:59:32.414 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T14:59:32.443 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T14:59:32.459 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T14:59:32.582 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2022-11-27T14:59:32.727 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T14:59:32.785 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T14:59:32.791 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2022-11-27T14:59:32.939 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T14:59:32.943 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T14:59:32.959 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T14:59:33.000 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T14:59:33.019 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T14:59:33.190 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2022-11-27T14:59:33.227 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2022-11-27T14:59:33.311 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T14:59:33.329 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2022-11-27T14:59:33.382 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T14:59:33.488 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T14:59:33.509 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T14:59:33.551 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T14:59:33.600 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T14:59:33.701 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:15.2.17-0.el8.x86_64 14/26 2022-11-27T14:59:33.755 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:15.2.17-0.el8.x86_64 14/26 2022-11-27T14:59:33.835 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T14:59:33.883 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:15.2.17-0.el8.x86_64 15/26 2022-11-27T14:59:33.971 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:15.2.17-0.el8.x86_64 15/26 2022-11-27T14:59:34.051 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T14:59:34.124 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T14:59:34.126 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T14:59:34.310 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2022-11-27T14:59:34.347 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T14:59:34.355 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T14:59:34.437 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T14:59:34.510 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/26 2022-11-27T14:59:34.559 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T14:59:34.614 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2022-11-27T14:59:34.623 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T14:59:34.634 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2022-11-27T14:59:34.776 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/26 2022-11-27T14:59:34.797 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/26 2022-11-27T14:59:34.851 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:15.2.17-0.el8.x86_64 20/26 2022-11-27T14:59:34.876 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T14:59:34.938 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2022-11-27T14:59:35.131 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:15.2.17-0.el8.x86_64 20/26 2022-11-27T14:59:35.223 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/26 2022-11-27T14:59:35.409 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T14:59:35.488 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T14:59:35.675 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2022-11-27T14:59:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T14:59:35.955 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T14:59:36.160 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:15.2.17-0.el8.x86_64 14/26 2022-11-27T14:59:36.351 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:15.2.17-0.el8.x86_64 15/26 2022-11-27T14:59:36.475 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T14:59:36.534 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2022-11-27T14:59:36.711 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2022-11-27T14:59:36.901 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/26 2022-11-27T14:59:37.068 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2022-11-27T14:59:37.234 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:15.2.17-0.el8.x86_64 20/26 2022-11-27T14:59:38.096 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T14:59:38.301 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T14:59:38.344 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T14:59:38.539 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T14:59:38.934 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T14:59:39.179 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T14:59:39.196 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T14:59:39.316 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T14:59:39.409 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T14:59:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T14:59:39.534 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T14:59:39.863 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T14:59:40.650 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T14:59:40.817 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T14:59:41.486 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2022-11-27T14:59:41.744 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T14:59:41.836 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:15.2.17-0.el8.x86_64 22/26 2022-11-27T14:59:41.958 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T14:59:58.314 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T14:59:58.314 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2022-11-27T14:59:58.314 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2022-11-27T14:59:58.315 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /dev 2022-11-27T14:59:58.315 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /run 2022-11-27T14:59:58.315 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2022-11-27T14:59:58.316 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2022-11-27T14:59:58.316 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2022-11-27T14:59:58.316 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2022-11-27T14:59:58.317 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /dev 2022-11-27T14:59:58.317 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T14:59:58.423 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T14:59:58.423 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2022-11-27T14:59:58.424 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2022-11-27T14:59:58.424 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2022-11-27T14:59:58.425 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /run 2022-11-27T14:59:58.425 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2022-11-27T14:59:58.425 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2022-11-27T14:59:58.425 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2022-11-27T14:59:58.426 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2022-11-27T14:59:58.426 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2022-11-27T14:59:58.426 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T14:59:59.393 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T14:59:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T14:59:59.456 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T14:59:59.457 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T14:59:59.499 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T14:59:59.499 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T14:59:59.658 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T14:59:59.659 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T14:59:59.679 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T14:59:59.679 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T15:00:00.251 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T15:00:00.252 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:15.2.17-0.el8.x86_64 1/26 2022-11-27T15:00:00.252 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:15.2.17-0.el8.x86_64 2/26 2022-11-27T15:00:00.252 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:15.2.17-0.el8.x86_64 3/26 2022-11-27T15:00:00.253 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T15:00:00.253 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T15:00:00.254 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T15:00:00.254 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T15:00:00.255 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 8/26 2022-11-27T15:00:00.255 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:15.2.17-0.el8.x86_64 9/26 2022-11-27T15:00:00.256 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T15:00:00.256 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:15.2.17-0.el8.x86_64 11/26 2022-11-27T15:00:00.257 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T15:00:00.257 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T15:00:00.258 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/26 2022-11-27T15:00:00.258 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2022-11-27T15:00:00.259 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2022-11-27T15:00:00.260 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/26 2022-11-27T15:00:00.260 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2022-11-27T15:00:00.260 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 19/26 2022-11-27T15:00:00.261 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2022-11-27T15:00:00.261 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2022-11-27T15:00:00.262 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2022-11-27T15:00:00.262 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T15:00:00.262 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2022-11-27T15:00:00.263 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:15.2.17-0.el8.x86_64 25/26 2022-11-27T15:00:00.325 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T15:00:00.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:15.2.17-0.el8.x86_64 1/26 2022-11-27T15:00:00.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:15.2.17-0.el8.x86_64 2/26 2022-11-27T15:00:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:15.2.17-0.el8.x86_64 3/26 2022-11-27T15:00:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T15:00:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T15:00:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T15:00:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T15:00:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 8/26 2022-11-27T15:00:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:15.2.17-0.el8.x86_64 9/26 2022-11-27T15:00:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T15:00:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:15.2.17-0.el8.x86_64 11/26 2022-11-27T15:00:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T15:00:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T15:00:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/26 2022-11-27T15:00:00.329 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2022-11-27T15:00:00.329 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2022-11-27T15:00:00.329 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/26 2022-11-27T15:00:00.330 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2022-11-27T15:00:00.330 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 19/26 2022-11-27T15:00:00.330 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2022-11-27T15:00:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2022-11-27T15:00:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2022-11-27T15:00:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T15:00:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2022-11-27T15:00:00.332 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:15.2.17-0.el8.x86_64 25/26 2022-11-27T15:00:01.110 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T15:00:01.111 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:01.111 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2022-11-27T15:00:01.112 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:15.2.17-0.el8.x86_64 librbd1-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.112 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2022-11-27T15:00:01.113 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.113 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.114 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.114 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.115 INFO:teuthology.orchestra.run.smithi102.stdout: fmt-6.2.1-1.el8.x86_64 2022-11-27T15:00:01.115 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T15:00:01.117 INFO:teuthology.orchestra.run.smithi102.stdout: leveldb-1.22-1.el8.x86_64 2022-11-27T15:00:01.117 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.118 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T15:00:01.118 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T15:00:01.119 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.119 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T15:00:01.120 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.121 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T15:00:01.121 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T15:00:01.122 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.122 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.122 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.123 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T15:00:01.124 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.124 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.125 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.126 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:01.126 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:01.289 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T15:00:01.290 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:01.290 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2022-11-27T15:00:01.290 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:15.2.17-0.el8.x86_64 librbd1-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.291 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2022-11-27T15:00:01.291 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.291 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.292 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.292 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.292 INFO:teuthology.orchestra.run.smithi133.stdout: fmt-6.2.1-1.el8.x86_64 2022-11-27T15:00:01.293 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T15:00:01.293 INFO:teuthology.orchestra.run.smithi133.stdout: leveldb-1.22-1.el8.x86_64 2022-11-27T15:00:01.293 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.293 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T15:00:01.294 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T15:00:01.294 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.294 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T15:00:01.294 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.295 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T15:00:01.295 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T15:00:01.295 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.295 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.296 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.296 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T15:00:01.296 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.296 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.297 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:01.297 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:01.297 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:01.373 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-test 2022-11-27T15:00:01.501 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-test 2022-11-27T15:00:01.763 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T15:00:01.763 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2022-11-27T15:00:01.764 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2022-11-27T15:00:01.764 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /dev 2022-11-27T15:00:01.764 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /run 2022-11-27T15:00:01.764 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2022-11-27T15:00:01.765 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2022-11-27T15:00:01.765 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2022-11-27T15:00:01.765 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2022-11-27T15:00:01.765 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /dev 2022-11-27T15:00:01.765 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:01.976 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:37 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:02.110 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:00:38 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:02.833 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T15:00:02.897 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2022-11-27T15:00:02.897 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T15:00:03.063 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:03.063 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:03.064 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2022-11-27T15:00:03.064 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:03.064 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2022-11-27T15:00:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:15.2.17-0.el8 ceph 42 M 2022-11-27T15:00:03.065 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2022-11-27T15:00:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2022-11-27T15:00:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2022-11-27T15:00:03.065 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2022-11-27T15:00:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2022-11-27T15:00:03.066 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:03.067 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T15:00:03.067 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:03.067 INFO:teuthology.orchestra.run.smithi102.stdout:Install 5 Packages 2022-11-27T15:00:03.067 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:03.069 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2022-11-27T15:00:03.069 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T15:00:03.069 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 43 M 2022-11-27T15:00:03.070 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 155 M 2022-11-27T15:00:03.070 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T15:00:03.157 INFO:teuthology.orchestra.run.smithi102.stdout:(1/5): jq-1.6-6.el8.x86_64.rpm 2.2 MB/s | 202 kB 00:00 2022-11-27T15:00:03.182 INFO:teuthology.orchestra.run.smithi102.stdout:(2/5): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2022-11-27T15:00:03.198 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:03.199 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:03.199 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2022-11-27T15:00:03.199 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:03.200 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2022-11-27T15:00:03.200 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:15.2.17-0.el8 ceph 42 M 2022-11-27T15:00:03.200 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2022-11-27T15:00:03.201 INFO:teuthology.orchestra.run.smithi133.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2022-11-27T15:00:03.201 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2022-11-27T15:00:03.201 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2022-11-27T15:00:03.201 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2022-11-27T15:00:03.202 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:03.202 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T15:00:03.202 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:03.202 INFO:teuthology.orchestra.run.smithi133.stdout:Install 5 Packages 2022-11-27T15:00:03.202 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:03.203 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 43 M 2022-11-27T15:00:03.204 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 155 M 2022-11-27T15:00:03.204 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T15:00:03.232 INFO:teuthology.orchestra.run.smithi102.stdout:(3/5): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2022-11-27T15:00:03.266 INFO:teuthology.orchestra.run.smithi102.stdout:(4/5): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2022-11-27T15:00:03.282 INFO:teuthology.orchestra.run.smithi133.stdout:(1/5): jq-1.6-6.el8.x86_64.rpm 2.5 MB/s | 202 kB 00:00 2022-11-27T15:00:03.316 INFO:teuthology.orchestra.run.smithi133.stdout:(2/5): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2022-11-27T15:00:03.360 INFO:teuthology.orchestra.run.smithi133.stdout:(3/5): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2022-11-27T15:00:03.416 INFO:teuthology.orchestra.run.smithi133.stdout:(4/5): socat-1.7.4.1-1.el8.x86_64.rpm 2.4 MB/s | 323 kB 00:00 2022-11-27T15:00:03.700 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T15:00:03.700 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:15.2.17-0.el8.x86_64 1/26 2022-11-27T15:00:03.700 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:15.2.17-0.el8.x86_64 2/26 2022-11-27T15:00:03.701 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:15.2.17-0.el8.x86_64 3/26 2022-11-27T15:00:03.701 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:15.2.17-0.el8.x86_64 4/26 2022-11-27T15:00:03.701 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:15.2.17-0.el8.x86_64 5/26 2022-11-27T15:00:03.701 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:15.2.17-0.el8.x86_64 6/26 2022-11-27T15:00:03.701 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:15.2.17-0.el8.x86_64 7/26 2022-11-27T15:00:03.702 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 8/26 2022-11-27T15:00:03.702 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:15.2.17-0.el8.x86_64 9/26 2022-11-27T15:00:03.702 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:15.2.17-0.el8.x86_64 10/26 2022-11-27T15:00:03.702 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:15.2.17-0.el8.x86_64 11/26 2022-11-27T15:00:03.703 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:15.2.17-0.el8.x86_64 12/26 2022-11-27T15:00:03.703 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:15.2.17-0.el8.x86_64 13/26 2022-11-27T15:00:03.703 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/26 2022-11-27T15:00:03.703 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2022-11-27T15:00:03.703 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2022-11-27T15:00:03.704 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/26 2022-11-27T15:00:03.704 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2022-11-27T15:00:03.704 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 19/26 2022-11-27T15:00:03.704 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2022-11-27T15:00:03.705 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2022-11-27T15:00:03.705 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2022-11-27T15:00:03.705 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:15.2.17-0.el8.x86_64 23/26 2022-11-27T15:00:03.705 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2022-11-27T15:00:03.705 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:15.2.17-0.el8.x86_64 25/26 2022-11-27T15:00:03.783 INFO:teuthology.orchestra.run.smithi102.stdout:(5/5): ceph-test-15.2.17-0.el8.x86_64.rpm 59 MB/s | 42 MB 00:00 2022-11-27T15:00:03.786 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:03.786 INFO:teuthology.orchestra.run.smithi102.stdout:Total 59 MB/s | 43 MB 00:00 2022-11-27T15:00:03.828 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T15:00:03.847 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T15:00:03.849 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T15:00:03.899 INFO:teuthology.orchestra.run.smithi133.stdout:(5/5): ceph-test-15.2.17-0.el8.x86_64.rpm 60 MB/s | 42 MB 00:00 2022-11-27T15:00:03.902 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:03.903 INFO:teuthology.orchestra.run.smithi133.stdout:Total 61 MB/s | 43 MB 00:00 2022-11-27T15:00:03.944 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T15:00:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T15:00:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T15:00:04.095 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T15:00:04.097 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T15:00:04.215 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T15:00:04.218 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T15:00:04.490 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T15:00:04.611 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T15:00:04.653 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2022-11-27T15:00:04.653 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:04.654 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2022-11-27T15:00:04.654 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:15.2.17-0.el8.x86_64 librbd1-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.655 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2022-11-27T15:00:04.655 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.655 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.656 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.656 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.656 INFO:teuthology.orchestra.run.smithi039.stdout: fmt-6.2.1-1.el8.x86_64 2022-11-27T15:00:04.656 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T15:00:04.657 INFO:teuthology.orchestra.run.smithi039.stdout: leveldb-1.22-1.el8.x86_64 2022-11-27T15:00:04.657 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.657 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T15:00:04.657 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T15:00:04.657 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.658 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T15:00:04.658 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.658 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T15:00:04.658 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T15:00:04.659 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.659 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.659 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.659 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T15:00:04.660 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.660 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.660 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:04.660 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:04.660 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:04.678 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/5 2022-11-27T15:00:04.807 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/5 2022-11-27T15:00:04.825 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/5 2022-11-27T15:00:04.879 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-test 2022-11-27T15:00:04.939 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T15:00:04.979 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/5 2022-11-27T15:00:05.021 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T15:00:05.142 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T15:00:05.212 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T15:00:05.514 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:39 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:06.679 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:06.680 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:06.681 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2022-11-27T15:00:06.681 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:06.681 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2022-11-27T15:00:06.681 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:15.2.17-0.el8 ceph 42 M 2022-11-27T15:00:06.682 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2022-11-27T15:00:06.682 INFO:teuthology.orchestra.run.smithi039.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2022-11-27T15:00:06.683 INFO:teuthology.orchestra.run.smithi039.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2022-11-27T15:00:06.683 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2022-11-27T15:00:06.683 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2022-11-27T15:00:06.684 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:06.684 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T15:00:06.684 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:06.684 INFO:teuthology.orchestra.run.smithi039.stdout:Install 5 Packages 2022-11-27T15:00:06.685 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:06.686 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 43 M 2022-11-27T15:00:06.686 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 155 M 2022-11-27T15:00:06.686 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T15:00:06.763 INFO:teuthology.orchestra.run.smithi039.stdout:(1/5): jq-1.6-6.el8.x86_64.rpm 2.5 MB/s | 202 kB 00:00 2022-11-27T15:00:06.796 INFO:teuthology.orchestra.run.smithi039.stdout:(2/5): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2022-11-27T15:00:06.821 INFO:teuthology.orchestra.run.smithi039.stdout:(3/5): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2022-11-27T15:00:06.847 INFO:teuthology.orchestra.run.smithi039.stdout:(4/5): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2022-11-27T15:00:07.346 INFO:teuthology.orchestra.run.smithi039.stdout:(5/5): ceph-test-15.2.17-0.el8.x86_64.rpm 63 MB/s | 42 MB 00:00 2022-11-27T15:00:07.349 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:07.350 INFO:teuthology.orchestra.run.smithi039.stdout:Total 64 MB/s | 43 MB 00:00 2022-11-27T15:00:07.390 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T15:00:07.411 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T15:00:07.411 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T15:00:07.674 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T15:00:07.677 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T15:00:08.082 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T15:00:08.285 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/5 2022-11-27T15:00:08.433 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/5 2022-11-27T15:00:08.571 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T15:00:08.651 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T15:00:08.889 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : jq-1.6-6.el8.x86_64 4/5 2022-11-27T15:00:09.103 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T15:00:09.106 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : jq-1.6-6.el8.x86_64 4/5 2022-11-27T15:00:09.331 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T15:00:09.356 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T15:00:09.356 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:15.2.17-0.el8.x86_64 1/5 2022-11-27T15:00:09.356 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : jq-1.6-6.el8.x86_64 2/5 2022-11-27T15:00:09.356 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T15:00:09.357 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/5 2022-11-27T15:00:09.580 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T15:00:09.581 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:15.2.17-0.el8.x86_64 1/5 2022-11-27T15:00:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : jq-1.6-6.el8.x86_64 2/5 2022-11-27T15:00:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T15:00:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/5 2022-11-27T15:00:09.806 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 5/5 2022-11-27T15:00:09.806 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:09.806 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2022-11-27T15:00:09.807 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:15.2.17-0.el8.x86_64 jq-1.6-6.el8.x86_64 2022-11-27T15:00:09.807 INFO:teuthology.orchestra.run.smithi102.stdout: oniguruma-6.8.2-2.el8.x86_64 socat-1.7.4.1-1.el8.x86_64 2022-11-27T15:00:09.807 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T15:00:09.807 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:09.808 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:10.044 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 5/5 2022-11-27T15:00:10.044 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:10.044 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2022-11-27T15:00:10.045 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:15.2.17-0.el8.x86_64 jq-1.6-6.el8.x86_64 2022-11-27T15:00:10.045 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma-6.8.2-2.el8.x86_64 socat-1.7.4.1-1.el8.x86_64 2022-11-27T15:00:10.045 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T15:00:10.045 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:10.046 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:10.062 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph 2022-11-27T15:00:10.247 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph 2022-11-27T15:00:10.666 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:46 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:10.855 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:00:46 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:11.748 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:11.752 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:11.752 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2022-11-27T15:00:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2022-11-27T15:00:11.753 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:15.2.17-0.el8 ceph 6.3 k 2022-11-27T15:00:11.754 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2022-11-27T15:00:11.754 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:15.2.17-0.el8 ceph 2.0 M 2022-11-27T15:00:11.755 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:15.2.17-0.el8 ceph 1.4 M 2022-11-27T15:00:11.755 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:15.2.17-0.el8 ceph-noarch 193 k 2022-11-27T15:00:11.756 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:15.2.17-0.el8 ceph 4.3 M 2022-11-27T15:00:11.756 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:15.2.17-0.el8 ceph 17 M 2022-11-27T15:00:11.756 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T15:00:11.757 INFO:teuthology.orchestra.run.smithi102.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T15:00:11.757 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2022-11-27T15:00:11.757 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T15:00:11.758 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T15:00:11.758 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2022-11-27T15:00:11.759 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T15:00:11.759 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T15:00:11.760 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T15:00:11.760 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2022-11-27T15:00:11.760 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2022-11-27T15:00:11.761 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T15:00:11.761 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T15:00:11.761 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T15:00:11.762 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2022-11-27T15:00:11.762 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2022-11-27T15:00:11.762 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2022-11-27T15:00:11.762 INFO:teuthology.orchestra.run.smithi102.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T15:00:11.763 INFO:teuthology.orchestra.run.smithi102.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T15:00:11.763 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T15:00:11.763 INFO:teuthology.orchestra.run.smithi102.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T15:00:11.763 INFO:teuthology.orchestra.run.smithi102.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T15:00:11.763 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T15:00:11.764 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T15:00:11.764 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2022-11-27T15:00:11.764 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T15:00:11.764 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T15:00:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:11.765 INFO:teuthology.orchestra.run.smithi102.stdout:Install 32 Packages 2022-11-27T15:00:11.766 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 28 M 2022-11-27T15:00:11.768 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 91 M 2022-11-27T15:00:11.769 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T15:00:11.807 INFO:teuthology.orchestra.run.smithi102.stdout:(1/32): ceph-15.2.17-0.el8.x86_64.rpm 128 kB/s | 6.3 kB 00:00 2022-11-27T15:00:11.849 INFO:teuthology.orchestra.run.smithi102.stdout:(2/32): ceph-mgr-15.2.17-0.el8.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2022-11-27T15:00:11.891 INFO:teuthology.orchestra.run.smithi102.stdout:(3/32): ceph-mds-15.2.17-0.el8.x86_64.rpm 15 MB/s | 2.0 MB 00:00 2022-11-27T15:00:11.916 INFO:teuthology.orchestra.run.smithi102.stdout:(4/32): ceph-mgr-modules-core-15.2.17-0.el8.noa 7.5 MB/s | 193 kB 00:00 2022-11-27T15:00:11.945 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:11.948 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:11.949 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2022-11-27T15:00:11.949 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:11.949 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2022-11-27T15:00:11.949 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:15.2.17-0.el8 ceph 6.3 k 2022-11-27T15:00:11.950 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2022-11-27T15:00:11.950 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:15.2.17-0.el8 ceph 2.0 M 2022-11-27T15:00:11.951 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:15.2.17-0.el8 ceph 1.4 M 2022-11-27T15:00:11.951 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:15.2.17-0.el8 ceph-noarch 193 k 2022-11-27T15:00:11.951 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:15.2.17-0.el8 ceph 4.3 M 2022-11-27T15:00:11.952 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:15.2.17-0.el8 ceph 17 M 2022-11-27T15:00:11.952 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T15:00:11.952 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T15:00:11.952 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2022-11-27T15:00:11.952 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T15:00:11.953 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T15:00:11.953 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2022-11-27T15:00:11.953 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T15:00:11.953 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T15:00:11.954 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T15:00:11.954 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2022-11-27T15:00:11.954 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2022-11-27T15:00:11.954 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T15:00:11.955 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T15:00:11.955 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T15:00:11.955 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2022-11-27T15:00:11.956 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2022-11-27T15:00:11.956 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2022-11-27T15:00:11.956 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T15:00:11.956 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T15:00:11.957 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T15:00:11.957 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T15:00:11.957 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T15:00:11.957 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T15:00:11.958 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T15:00:11.958 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2022-11-27T15:00:11.958 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T15:00:11.959 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:11.959 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T15:00:11.960 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:11.960 INFO:teuthology.orchestra.run.smithi133.stdout:Install 32 Packages 2022-11-27T15:00:11.960 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:11.961 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 28 M 2022-11-27T15:00:11.961 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 91 M 2022-11-27T15:00:11.961 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T15:00:12.000 INFO:teuthology.orchestra.run.smithi102.stdout:(5/32): ceph-mon-15.2.17-0.el8.x86_64.rpm 23 MB/s | 4.3 MB 00:00 2022-11-27T15:00:12.014 INFO:teuthology.orchestra.run.smithi133.stdout:(1/32): ceph-15.2.17-0.el8.x86_64.rpm 106 kB/s | 6.3 kB 00:00 2022-11-27T15:00:12.033 INFO:teuthology.orchestra.run.smithi102.stdout:(6/32): python3-mako-1.0.6-13.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2022-11-27T15:00:12.065 INFO:teuthology.orchestra.run.smithi133.stdout:(2/32): ceph-mds-15.2.17-0.el8.x86_64.rpm 18 MB/s | 2.0 MB 00:00 2022-11-27T15:00:12.067 INFO:teuthology.orchestra.run.smithi102.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 3.1 MB/s | 103 kB 00:00 2022-11-27T15:00:12.093 INFO:teuthology.orchestra.run.smithi102.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 424 kB/s | 39 kB 00:00 2022-11-27T15:00:12.107 INFO:teuthology.orchestra.run.smithi133.stdout:(3/32): ceph-mgr-15.2.17-0.el8.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2022-11-27T15:00:12.118 INFO:teuthology.orchestra.run.smithi102.stdout:(9/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2022-11-27T15:00:12.141 INFO:teuthology.orchestra.run.smithi133.stdout:(4/32): ceph-mgr-modules-core-15.2.17-0.el8.noa 5.7 MB/s | 193 kB 00:00 2022-11-27T15:00:12.172 INFO:teuthology.orchestra.run.smithi102.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.3 MB/s | 237 kB 00:00 2022-11-27T15:00:12.202 INFO:teuthology.orchestra.run.smithi102.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 4.1 MB/s | 457 kB 00:00 2022-11-27T15:00:12.216 INFO:teuthology.orchestra.run.smithi133.stdout:(5/32): ceph-mon-15.2.17-0.el8.x86_64.rpm 22 MB/s | 4.3 MB 00:00 2022-11-27T15:00:12.219 INFO:teuthology.orchestra.run.smithi102.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 6.3 MB/s | 109 kB 00:00 2022-11-27T15:00:12.241 INFO:teuthology.orchestra.run.smithi133.stdout:(6/32): python3-mako-1.0.6-13.el8.noarch.rpm 1.5 MB/s | 157 kB 00:00 2022-11-27T15:00:12.252 INFO:teuthology.orchestra.run.smithi102.stdout:(13/32): python3-cryptography-3.2.1-5.el8.x86_6 6.8 MB/s | 559 kB 00:00 2022-11-27T15:00:12.283 INFO:teuthology.orchestra.run.smithi133.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 2.4 MB/s | 103 kB 00:00 2022-11-27T15:00:12.308 INFO:teuthology.orchestra.run.smithi133.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2022-11-27T15:00:12.333 INFO:teuthology.orchestra.run.smithi133.stdout:(9/32): python3-markupsafe-0.23-19.el8.x86_64.r 333 kB/s | 39 kB 00:00 2022-11-27T15:00:12.358 INFO:teuthology.orchestra.run.smithi133.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 8.9 MB/s | 457 kB 00:00 2022-11-27T15:00:12.376 INFO:teuthology.orchestra.run.smithi133.stdout:(11/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.5 MB/s | 237 kB 00:00 2022-11-27T15:00:12.401 INFO:teuthology.orchestra.run.smithi133.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2022-11-27T15:00:12.426 INFO:teuthology.orchestra.run.smithi133.stdout:(13/32): python3-cryptography-3.2.1-5.el8.x86_6 8.1 MB/s | 559 kB 00:00 2022-11-27T15:00:12.453 INFO:teuthology.orchestra.run.smithi102.stdout:(14/32): ceph-osd-15.2.17-0.el8.x86_64.rpm 28 MB/s | 17 MB 00:00 2022-11-27T15:00:12.470 INFO:teuthology.orchestra.run.smithi102.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 174 kB/s | 44 kB 00:00 2022-11-27T15:00:12.495 INFO:teuthology.orchestra.run.smithi102.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 765 kB/s | 185 kB 00:00 2022-11-27T15:00:12.554 INFO:teuthology.orchestra.run.smithi102.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2022-11-27T15:00:12.612 INFO:teuthology.orchestra.run.smithi102.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 301 kB/s | 18 kB 00:00 2022-11-27T15:00:12.638 INFO:teuthology.orchestra.run.smithi102.stdout:(19/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 937 kB/s | 173 kB 00:00 2022-11-27T15:00:12.652 INFO:teuthology.orchestra.run.smithi133.stdout:(14/32): ceph-osd-15.2.17-0.el8.x86_64.rpm 29 MB/s | 17 MB 00:00 2022-11-27T15:00:12.663 INFO:teuthology.orchestra.run.smithi102.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.9 MB/s | 384 kB 00:00 2022-11-27T15:00:12.676 INFO:teuthology.orchestra.run.smithi133.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 158 kB/s | 44 kB 00:00 2022-11-27T15:00:12.693 INFO:teuthology.orchestra.run.smithi133.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 693 kB/s | 185 kB 00:00 2022-11-27T15:00:12.713 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : jq-1.6-6.el8.x86_64 4/5 2022-11-27T15:00:12.713 INFO:teuthology.orchestra.run.smithi102.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 780 kB/s | 59 kB 00:00 2022-11-27T15:00:12.730 INFO:teuthology.orchestra.run.smithi102.stdout:(22/32): python3-logutils-0.3.5-11.el8.noarch.r 415 kB/s | 49 kB 00:00 2022-11-27T15:00:12.744 INFO:teuthology.orchestra.run.smithi133.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2022-11-27T15:00:12.764 INFO:teuthology.orchestra.run.smithi102.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.7 MB/s | 283 kB 00:00 2022-11-27T15:00:12.794 INFO:teuthology.orchestra.run.smithi133.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 352 kB/s | 18 kB 00:00 2022-11-27T15:00:12.809 INFO:teuthology.orchestra.run.smithi102.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 163 kB/s | 16 kB 00:00 2022-11-27T15:00:12.819 INFO:teuthology.orchestra.run.smithi133.stdout:(19/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.0 MB/s | 173 kB 00:00 2022-11-27T15:00:12.831 INFO:teuthology.orchestra.run.smithi102.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 186 kB/s | 19 kB 00:00 2022-11-27T15:00:12.845 INFO:teuthology.orchestra.run.smithi133.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 965 kB/s | 49 kB 00:00 2022-11-27T15:00:12.847 INFO:teuthology.orchestra.run.smithi102.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 286 kB/s | 24 kB 00:00 2022-11-27T15:00:12.870 INFO:teuthology.orchestra.run.smithi133.stdout:(21/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.9 MB/s | 384 kB 00:00 2022-11-27T15:00:12.895 INFO:teuthology.orchestra.run.smithi133.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 774 kB/s | 59 kB 00:00 2022-11-27T15:00:12.902 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T15:00:12.906 INFO:teuthology.orchestra.run.smithi102.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 366 kB/s | 27 kB 00:00 2022-11-27T15:00:12.921 INFO:teuthology.orchestra.run.smithi133.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2022-11-27T15:00:12.932 INFO:teuthology.orchestra.run.smithi102.stdout:(28/32): python3-tempora-1.14.1-5.el8.noarch.rp 291 kB/s | 29 kB 00:00 2022-11-27T15:00:12.946 INFO:teuthology.orchestra.run.smithi133.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 206 kB/s | 16 kB 00:00 2022-11-27T15:00:12.966 INFO:teuthology.orchestra.run.smithi102.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.0 MB/s | 241 kB 00:00 2022-11-27T15:00:12.971 INFO:teuthology.orchestra.run.smithi133.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 247 kB/s | 19 kB 00:00 2022-11-27T15:00:12.990 INFO:teuthology.orchestra.run.smithi102.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.4 MB/s | 85 kB 00:00 2022-11-27T15:00:12.996 INFO:teuthology.orchestra.run.smithi133.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 317 kB/s | 24 kB 00:00 2022-11-27T15:00:13.016 INFO:teuthology.orchestra.run.smithi102.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.2 MB/s | 251 kB 00:00 2022-11-27T15:00:13.021 INFO:teuthology.orchestra.run.smithi133.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 387 kB/s | 29 kB 00:00 2022-11-27T15:00:13.041 INFO:teuthology.orchestra.run.smithi102.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 304 kB/s | 23 kB 00:00 2022-11-27T15:00:13.044 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:13.045 INFO:teuthology.orchestra.run.smithi102.stdout:Total 22 MB/s | 28 MB 00:01 2022-11-27T15:00:13.047 INFO:teuthology.orchestra.run.smithi133.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 364 kB/s | 27 kB 00:00 2022-11-27T15:00:13.072 INFO:teuthology.orchestra.run.smithi133.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 3.1 MB/s | 241 kB 00:00 2022-11-27T15:00:13.089 INFO:teuthology.orchestra.run.smithi133.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 2.0 MB/s | 85 kB 00:00 2022-11-27T15:00:13.120 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T15:00:13.122 INFO:teuthology.orchestra.run.smithi133.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 455 kB/s | 23 kB 00:00 2022-11-27T15:00:13.141 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-test-2:15.2.17-0.el8.x86_64 5/5 2022-11-27T15:00:13.141 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:15.2.17-0.el8.x86_64 1/5 2022-11-27T15:00:13.142 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : jq-1.6-6.el8.x86_64 2/5 2022-11-27T15:00:13.142 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 3/5 2022-11-27T15:00:13.142 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/5 2022-11-27T15:00:13.148 INFO:teuthology.orchestra.run.smithi133.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.9 MB/s | 251 kB 00:00 2022-11-27T15:00:13.152 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:13.152 INFO:teuthology.orchestra.run.smithi133.stdout:Total 24 MB/s | 28 MB 00:01 2022-11-27T15:00:13.157 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T15:00:13.158 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T15:00:13.227 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T15:00:13.263 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T15:00:13.264 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T15:00:13.436 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T15:00:13.439 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T15:00:13.539 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T15:00:13.542 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T15:00:13.623 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 5/5 2022-11-27T15:00:13.624 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:13.624 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2022-11-27T15:00:13.624 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:15.2.17-0.el8.x86_64 jq-1.6-6.el8.x86_64 2022-11-27T15:00:13.624 INFO:teuthology.orchestra.run.smithi039.stdout: oniguruma-6.8.2-2.el8.x86_64 socat-1.7.4.1-1.el8.x86_64 2022-11-27T15:00:13.625 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T15:00:13.625 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:13.625 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:13.794 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph 2022-11-27T15:00:14.419 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:48 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:14.622 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T15:00:14.717 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T15:00:14.842 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2022-11-27T15:00:14.937 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2022-11-27T15:00:15.012 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2022-11-27T15:00:15.131 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2022-11-27T15:00:15.207 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2022-11-27T15:00:15.351 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2022-11-27T15:00:15.411 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2022-11-27T15:00:15.563 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2022-11-27T15:00:15.566 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2022-11-27T15:00:15.580 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:15.583 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:15.584 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2022-11-27T15:00:15.584 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:15.584 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2022-11-27T15:00:15.584 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:15.2.17-0.el8 ceph 6.3 k 2022-11-27T15:00:15.585 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2022-11-27T15:00:15.585 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:15.2.17-0.el8 ceph 2.0 M 2022-11-27T15:00:15.585 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:15.2.17-0.el8 ceph 1.4 M 2022-11-27T15:00:15.586 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:15.2.17-0.el8 ceph-noarch 193 k 2022-11-27T15:00:15.586 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:15.2.17-0.el8 ceph 4.3 M 2022-11-27T15:00:15.586 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:15.2.17-0.el8 ceph 17 M 2022-11-27T15:00:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T15:00:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T15:00:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2022-11-27T15:00:15.587 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T15:00:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T15:00:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2022-11-27T15:00:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T15:00:15.588 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T15:00:15.589 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T15:00:15.589 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2022-11-27T15:00:15.589 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2022-11-27T15:00:15.589 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T15:00:15.590 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T15:00:15.590 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T15:00:15.590 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2022-11-27T15:00:15.590 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2022-11-27T15:00:15.591 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2022-11-27T15:00:15.591 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T15:00:15.591 INFO:teuthology.orchestra.run.smithi039.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T15:00:15.591 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T15:00:15.592 INFO:teuthology.orchestra.run.smithi039.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T15:00:15.592 INFO:teuthology.orchestra.run.smithi039.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T15:00:15.592 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T15:00:15.592 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T15:00:15.592 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2022-11-27T15:00:15.593 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T15:00:15.593 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:15.593 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T15:00:15.593 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:15.594 INFO:teuthology.orchestra.run.smithi039.stdout:Install 32 Packages 2022-11-27T15:00:15.594 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:15.595 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 28 M 2022-11-27T15:00:15.595 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 91 M 2022-11-27T15:00:15.595 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T15:00:15.646 INFO:teuthology.orchestra.run.smithi039.stdout:(1/32): ceph-15.2.17-0.el8.x86_64.rpm 113 kB/s | 6.3 kB 00:00 2022-11-27T15:00:15.696 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2022-11-27T15:00:15.696 INFO:teuthology.orchestra.run.smithi039.stdout:(2/32): ceph-mds-15.2.17-0.el8.x86_64.rpm 19 MB/s | 2.0 MB 00:00 2022-11-27T15:00:15.720 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2022-11-27T15:00:15.755 INFO:teuthology.orchestra.run.smithi039.stdout:(3/32): ceph-mgr-15.2.17-0.el8.x86_64.rpm 8.7 MB/s | 1.4 MB 00:00 2022-11-27T15:00:15.772 INFO:teuthology.orchestra.run.smithi039.stdout:(4/32): ceph-mgr-modules-core-15.2.17-0.el8.noa 12 MB/s | 193 kB 00:00 2022-11-27T15:00:15.840 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2022-11-27T15:00:15.864 INFO:teuthology.orchestra.run.smithi039.stdout:(5/32): ceph-mon-15.2.17-0.el8.x86_64.rpm 20 MB/s | 4.3 MB 00:00 2022-11-27T15:00:15.865 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2022-11-27T15:00:15.897 INFO:teuthology.orchestra.run.smithi039.stdout:(6/32): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2022-11-27T15:00:15.923 INFO:teuthology.orchestra.run.smithi039.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.2 MB/s | 103 kB 00:00 2022-11-27T15:00:15.939 INFO:teuthology.orchestra.run.smithi039.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 520 kB/s | 39 kB 00:00 2022-11-27T15:00:15.956 INFO:teuthology.orchestra.run.smithi039.stdout:(9/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2022-11-27T15:00:15.959 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2022-11-27T15:00:15.982 INFO:teuthology.orchestra.run.smithi039.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.2 MB/s | 237 kB 00:00 2022-11-27T15:00:16.007 INFO:teuthology.orchestra.run.smithi039.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 6.6 MB/s | 457 kB 00:00 2022-11-27T15:00:16.009 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2022-11-27T15:00:16.032 INFO:teuthology.orchestra.run.smithi039.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2022-11-27T15:00:16.057 INFO:teuthology.orchestra.run.smithi039.stdout:(13/32): python3-cryptography-3.2.1-5.el8.x86_6 7.3 MB/s | 559 kB 00:00 2022-11-27T15:00:16.166 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2022-11-27T15:00:16.170 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2022-11-27T15:00:16.266 INFO:teuthology.orchestra.run.smithi039.stdout:(14/32): ceph-osd-15.2.17-0.el8.x86_64.rpm 29 MB/s | 17 MB 00:00 2022-11-27T15:00:16.283 INFO:teuthology.orchestra.run.smithi039.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 174 kB/s | 44 kB 00:00 2022-11-27T15:00:16.308 INFO:teuthology.orchestra.run.smithi039.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 739 kB/s | 185 kB 00:00 2022-11-27T15:00:16.351 INFO:teuthology.orchestra.run.smithi039.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 265 kB/s | 11 kB 00:00 2022-11-27T15:00:16.353 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2022-11-27T15:00:16.360 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2022-11-27T15:00:16.392 INFO:teuthology.orchestra.run.smithi039.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 423 kB/s | 18 kB 00:00 2022-11-27T15:00:16.418 INFO:teuthology.orchestra.run.smithi039.stdout:(19/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.8 MB/s | 384 kB 00:00 2022-11-27T15:00:16.443 INFO:teuthology.orchestra.run.smithi039.stdout:(20/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 978 kB/s | 173 kB 00:00 2022-11-27T15:00:16.468 INFO:teuthology.orchestra.run.smithi039.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 643 kB/s | 49 kB 00:00 2022-11-27T15:00:16.493 INFO:teuthology.orchestra.run.smithi039.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 777 kB/s | 59 kB 00:00 2022-11-27T15:00:16.519 INFO:teuthology.orchestra.run.smithi039.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2022-11-27T15:00:16.550 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2022-11-27T15:00:16.565 INFO:teuthology.orchestra.run.smithi039.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.3 MB/s | 283 kB 00:00 2022-11-27T15:00:16.573 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2022-11-27T15:00:16.594 INFO:teuthology.orchestra.run.smithi039.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 186 kB/s | 19 kB 00:00 2022-11-27T15:00:16.620 INFO:teuthology.orchestra.run.smithi039.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 238 kB/s | 24 kB 00:00 2022-11-27T15:00:16.645 INFO:teuthology.orchestra.run.smithi039.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 370 kB/s | 29 kB 00:00 2022-11-27T15:00:16.670 INFO:teuthology.orchestra.run.smithi039.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 549 kB/s | 27 kB 00:00 2022-11-27T15:00:16.695 INFO:teuthology.orchestra.run.smithi039.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 4.9 MB/s | 251 kB 00:00 2022-11-27T15:00:16.721 INFO:teuthology.orchestra.run.smithi039.stdout:(30/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.3 MB/s | 241 kB 00:00 2022-11-27T15:00:16.738 INFO:teuthology.orchestra.run.smithi039.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.2 MB/s | 85 kB 00:00 2022-11-27T15:00:16.757 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2022-11-27T15:00:16.763 INFO:teuthology.orchestra.run.smithi039.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 343 kB/s | 23 kB 00:00 2022-11-27T15:00:16.767 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:16.767 INFO:teuthology.orchestra.run.smithi039.stdout:Total 24 MB/s | 28 MB 00:01 2022-11-27T15:00:16.810 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2022-11-27T15:00:16.847 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T15:00:16.889 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T15:00:16.889 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T15:00:17.002 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2022-11-27T15:00:17.126 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2022-11-27T15:00:17.173 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T15:00:17.176 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T15:00:17.370 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2022-11-27T15:00:17.398 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2022-11-27T15:00:17.567 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2022-11-27T15:00:17.609 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2022-11-27T15:00:17.789 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2022-11-27T15:00:17.799 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2022-11-27T15:00:18.029 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T15:00:18.063 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2022-11-27T15:00:18.274 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T15:00:18.328 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2022-11-27T15:00:18.331 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T15:00:18.518 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2022-11-27T15:00:18.539 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2022-11-27T15:00:18.545 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2022-11-27T15:00:18.706 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2022-11-27T15:00:18.737 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2022-11-27T15:00:18.744 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2022-11-27T15:00:18.900 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2022-11-27T15:00:18.908 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2022-11-27T15:00:18.970 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2022-11-27T15:00:19.119 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2022-11-27T15:00:19.158 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2022-11-27T15:00:19.184 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2022-11-27T15:00:19.341 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2022-11-27T15:00:19.402 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2022-11-27T15:00:19.415 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2022-11-27T15:00:19.488 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2022-11-27T15:00:19.601 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2022-11-27T15:00:19.630 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2022-11-27T15:00:19.638 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2022-11-27T15:00:19.750 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2022-11-27T15:00:19.849 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2022-11-27T15:00:19.909 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2022-11-27T15:00:20.013 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2022-11-27T15:00:20.145 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2022-11-27T15:00:20.185 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2022-11-27T15:00:20.331 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2022-11-27T15:00:20.342 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2022-11-27T15:00:20.508 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2022-11-27T15:00:20.520 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2022-11-27T15:00:20.701 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 27/32 2022-11-27T15:00:20.832 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 27/32 2022-11-27T15:00:20.857 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T15:00:20.882 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2022-11-27T15:00:21.020 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T15:00:21.132 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2022-11-27T15:00:21.284 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2022-11-27T15:00:21.514 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2022-11-27T15:00:21.746 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T15:00:22.044 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2022-11-27T15:00:22.217 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2022-11-27T15:00:22.220 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T15:00:22.382 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T15:00:22.405 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2022-11-27T15:00:22.412 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T15:00:22.540 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T15:00:22.625 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2022-11-27T15:00:22.787 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T15:00:22.913 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T15:00:22.930 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2022-11-27T15:00:22.954 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T15:00:23.056 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T15:00:23.136 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T15:00:23.148 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2022-11-27T15:00:23.239 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T15:00:23.287 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T15:00:23.304 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T15:00:23.367 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2022-11-27T15:00:23.381 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T15:00:23.393 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T15:00:23.398 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T15:00:23.488 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T15:00:23.730 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2022-11-27T15:00:23.772 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T15:00:23.772 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:15.2.17-0.el8.x86_64 1/32 2022-11-27T15:00:23.773 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:15.2.17-0.el8.x86_64 2/32 2022-11-27T15:00:23.773 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:15.2.17-0.el8.x86_64 3/32 2022-11-27T15:00:23.773 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:15.2.17-0.el8.x86_64 4/32 2022-11-27T15:00:23.777 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:15.2.17-0.el8.x86_64 5/32 2022-11-27T15:00:23.777 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 6/32 2022-11-27T15:00:23.778 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2022-11-27T15:00:23.778 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2022-11-27T15:00:23.778 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2022-11-27T15:00:23.779 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2022-11-27T15:00:23.779 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2022-11-27T15:00:23.779 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2022-11-27T15:00:23.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2022-11-27T15:00:23.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2022-11-27T15:00:23.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2022-11-27T15:00:23.780 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2022-11-27T15:00:23.781 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T15:00:23.781 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2022-11-27T15:00:23.781 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2022-11-27T15:00:23.781 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2022-11-27T15:00:23.781 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2022-11-27T15:00:23.782 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2022-11-27T15:00:23.782 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2022-11-27T15:00:23.782 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2022-11-27T15:00:23.782 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2022-11-27T15:00:23.783 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2022-11-27T15:00:23.783 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2022-11-27T15:00:23.783 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2022-11-27T15:00:23.783 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2022-11-27T15:00:23.784 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2022-11-27T15:00:23.784 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2022-11-27T15:00:23.865 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T15:00:23.865 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:15.2.17-0.el8.x86_64 1/32 2022-11-27T15:00:23.866 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:15.2.17-0.el8.x86_64 2/32 2022-11-27T15:00:23.866 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:15.2.17-0.el8.x86_64 3/32 2022-11-27T15:00:23.866 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:15.2.17-0.el8.x86_64 4/32 2022-11-27T15:00:23.866 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:15.2.17-0.el8.x86_64 5/32 2022-11-27T15:00:23.867 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 6/32 2022-11-27T15:00:23.867 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2022-11-27T15:00:23.867 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2022-11-27T15:00:23.867 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2022-11-27T15:00:23.868 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2022-11-27T15:00:23.868 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2022-11-27T15:00:23.868 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2022-11-27T15:00:23.868 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2022-11-27T15:00:23.868 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2022-11-27T15:00:23.869 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2022-11-27T15:00:23.869 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2022-11-27T15:00:23.869 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T15:00:23.869 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2022-11-27T15:00:23.870 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2022-11-27T15:00:23.870 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2022-11-27T15:00:23.870 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2022-11-27T15:00:23.870 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2022-11-27T15:00:23.870 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2022-11-27T15:00:23.871 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2022-11-27T15:00:23.871 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2022-11-27T15:00:23.871 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2022-11-27T15:00:23.871 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2022-11-27T15:00:23.872 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2022-11-27T15:00:23.872 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2022-11-27T15:00:23.873 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2022-11-27T15:00:23.873 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2022-11-27T15:00:24.024 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2022-11-27T15:00:24.382 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 27/32 2022-11-27T15:00:24.529 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T15:00:24.709 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2022-11-27T15:00:24.709 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:24.709 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2022-11-27T15:00:24.710 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:24.710 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:24.710 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:24.711 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 2022-11-27T15:00:24.711 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:24.711 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:24.711 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T15:00:24.712 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T15:00:24.712 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T15:00:24.712 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T15:00:24.712 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T15:00:24.713 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T15:00:24.713 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T15:00:24.713 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T15:00:24.713 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T15:00:24.714 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T15:00:24.714 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T15:00:24.714 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T15:00:24.714 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T15:00:24.715 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T15:00:24.715 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T15:00:24.715 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T15:00:24.715 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T15:00:24.716 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T15:00:24.716 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T15:00:24.716 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T15:00:24.717 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T15:00:24.717 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T15:00:24.717 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T15:00:24.717 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T15:00:24.718 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T15:00:24.718 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T15:00:24.718 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:24.718 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:24.719 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2022-11-27T15:00:24.719 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:24.720 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2022-11-27T15:00:24.720 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:24.720 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:24.720 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:24.721 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 2022-11-27T15:00:24.721 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:24.721 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:24.721 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T15:00:24.722 INFO:teuthology.orchestra.run.smithi102.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T15:00:24.722 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T15:00:24.722 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T15:00:24.722 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T15:00:24.723 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T15:00:24.723 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T15:00:24.723 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T15:00:24.723 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T15:00:24.724 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T15:00:24.724 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T15:00:24.724 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T15:00:24.724 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T15:00:24.725 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T15:00:24.725 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T15:00:24.725 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T15:00:24.725 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T15:00:24.726 INFO:teuthology.orchestra.run.smithi102.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T15:00:24.726 INFO:teuthology.orchestra.run.smithi102.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T15:00:24.726 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T15:00:24.726 INFO:teuthology.orchestra.run.smithi102.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T15:00:24.727 INFO:teuthology.orchestra.run.smithi102.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T15:00:24.727 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T15:00:24.727 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T15:00:24.727 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T15:00:24.728 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T15:00:24.728 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:24.728 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:24.908 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-base 2022-11-27T15:00:24.969 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-base 2022-11-27T15:00:25.508 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:01 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:25.569 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:01 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:25.977 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2022-11-27T15:00:26.136 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T15:00:26.504 INFO:teuthology.orchestra.run.smithi133.stdout:Package ceph-base-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:26.532 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2022-11-27T15:00:26.563 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-base-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:26.595 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:26.596 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T15:00:26.596 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:26.656 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:26.657 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T15:00:26.658 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:26.677 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T15:00:26.679 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-immutable-object-cache 2022-11-27T15:00:26.751 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-immutable-object-cache 2022-11-27T15:00:26.870 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2022-11-27T15:00:26.994 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T15:00:27.012 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2022-11-27T15:00:27.117 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T15:00:27.277 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:03 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:27.350 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:03 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:27.521 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-2:15.2.17-0.el8.x86_64 32/32 2022-11-27T15:00:27.521 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:15.2.17-0.el8.x86_64 1/32 2022-11-27T15:00:27.521 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:15.2.17-0.el8.x86_64 2/32 2022-11-27T15:00:27.522 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:15.2.17-0.el8.x86_64 3/32 2022-11-27T15:00:27.522 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:15.2.17-0.el8.x86_64 4/32 2022-11-27T15:00:27.522 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:15.2.17-0.el8.x86_64 5/32 2022-11-27T15:00:27.523 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 6/32 2022-11-27T15:00:27.523 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2022-11-27T15:00:27.523 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2022-11-27T15:00:27.523 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2022-11-27T15:00:27.524 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2022-11-27T15:00:27.524 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2022-11-27T15:00:27.524 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2022-11-27T15:00:27.524 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2022-11-27T15:00:27.525 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2022-11-27T15:00:27.525 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2022-11-27T15:00:27.525 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2022-11-27T15:00:27.526 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-11-27T15:00:27.526 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2022-11-27T15:00:27.526 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2022-11-27T15:00:27.526 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2022-11-27T15:00:27.527 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2022-11-27T15:00:27.527 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2022-11-27T15:00:27.527 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2022-11-27T15:00:27.528 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2022-11-27T15:00:27.528 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2022-11-27T15:00:27.528 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2022-11-27T15:00:27.528 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2022-11-27T15:00:27.528 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2022-11-27T15:00:27.529 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2022-11-27T15:00:27.530 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2022-11-27T15:00:27.530 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2022-11-27T15:00:28.368 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:28.370 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:28.370 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repo Size 2022-11-27T15:00:28.371 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:28.371 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2022-11-27T15:00:28.371 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:15.2.17-0.el8 ceph 203 k 2022-11-27T15:00:28.372 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:28.372 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T15:00:28.372 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:28.372 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2022-11-27T15:00:28.373 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:28.373 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 203 k 2022-11-27T15:00:28.373 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 599 k 2022-11-27T15:00:28.373 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T15:00:28.420 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-immutable-object-cache-15.2.17-0.el8.x86_6 4.1 MB/s | 203 kB 00:00 2022-11-27T15:00:28.420 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:28.421 INFO:teuthology.orchestra.run.smithi133.stdout:Total 4.0 MB/s | 203 kB 00:00 2022-11-27T15:00:28.421 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T15:00:28.430 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T15:00:28.431 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T15:00:28.432 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:28.432 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:28.433 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2022-11-27T15:00:28.433 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:28.433 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2022-11-27T15:00:28.433 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:15.2.17-0.el8 ceph 203 k 2022-11-27T15:00:28.434 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:28.434 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T15:00:28.434 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:28.435 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2022-11-27T15:00:28.435 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:28.435 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 203 k 2022-11-27T15:00:28.436 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 599 k 2022-11-27T15:00:28.436 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T15:00:28.445 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2022-11-27T15:00:28.445 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:28.445 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2022-11-27T15:00:28.445 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:28.446 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:28.446 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:28.446 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 2022-11-27T15:00:28.446 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:28.447 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:28.447 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T15:00:28.447 INFO:teuthology.orchestra.run.smithi039.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T15:00:28.447 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T15:00:28.448 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T15:00:28.448 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T15:00:28.448 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T15:00:28.448 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T15:00:28.449 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T15:00:28.449 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T15:00:28.449 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T15:00:28.449 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T15:00:28.450 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T15:00:28.450 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T15:00:28.450 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T15:00:28.450 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T15:00:28.451 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T15:00:28.451 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T15:00:28.451 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T15:00:28.451 INFO:teuthology.orchestra.run.smithi039.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T15:00:28.452 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T15:00:28.452 INFO:teuthology.orchestra.run.smithi039.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T15:00:28.452 INFO:teuthology.orchestra.run.smithi039.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T15:00:28.452 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T15:00:28.453 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T15:00:28.453 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T15:00:28.453 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T15:00:28.453 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:28.454 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:28.478 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-immutable-object-cache-15.2.17-0.el8.x86_6 4.4 MB/s | 203 kB 00:00 2022-11-27T15:00:28.479 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T15:00:28.480 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:28.480 INFO:teuthology.orchestra.run.smithi102.stdout:Total 4.3 MB/s | 203 kB 00:00 2022-11-27T15:00:28.481 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T15:00:28.482 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T15:00:28.490 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T15:00:28.490 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T15:00:28.537 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T15:00:28.540 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T15:00:28.591 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T15:00:28.654 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T15:00:28.661 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-base 2022-11-27T15:00:28.755 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:28.836 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:29.133 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:29.293 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:03 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:29.560 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:29.560 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:29.561 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2022-11-27T15:00:29.561 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:29.561 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:29.562 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:29.648 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:29.648 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:29.648 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2022-11-27T15:00:29.648 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:29.649 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:29.649 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:29.785 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-mgr 2022-11-27T15:00:29.859 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr 2022-11-27T15:00:30.355 INFO:teuthology.orchestra.run.smithi039.stdout:Package ceph-base-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:30.397 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:06 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:30.456 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:30.457 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T15:00:30.458 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:30.471 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:06 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:30.540 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-immutable-object-cache 2022-11-27T15:00:31.170 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:05 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:31.388 INFO:teuthology.orchestra.run.smithi133.stdout:Package ceph-mgr-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:31.470 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-mgr-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:31.477 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:31.479 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T15:00:31.479 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:31.559 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:31.560 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T15:00:31.561 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:31.565 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-fuse 2022-11-27T15:00:31.647 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-fuse 2022-11-27T15:00:32.164 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:08 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:32.243 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:08 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:32.340 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:32.341 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:32.341 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repo Size 2022-11-27T15:00:32.342 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:32.342 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2022-11-27T15:00:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:15.2.17-0.el8 ceph 203 k 2022-11-27T15:00:32.342 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:32.343 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T15:00:32.343 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:32.343 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2022-11-27T15:00:32.343 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:32.344 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 203 k 2022-11-27T15:00:32.344 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 599 k 2022-11-27T15:00:32.344 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T15:00:32.379 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-immutable-object-cache-15.2.17-0.el8.x86_6 5.4 MB/s | 203 kB 00:00 2022-11-27T15:00:32.380 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:32.380 INFO:teuthology.orchestra.run.smithi039.stdout:Total 5.3 MB/s | 203 kB 00:00 2022-11-27T15:00:32.380 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T15:00:32.392 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T15:00:32.393 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T15:00:32.445 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T15:00:32.448 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T15:00:32.569 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T15:00:32.709 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:33.099 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:33.235 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:33.237 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:33.237 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:33.237 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:33.238 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2022-11-27T15:00:33.238 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:15.2.17-0.el8 ceph 572 k 2022-11-27T15:00:33.238 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2022-11-27T15:00:33.239 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2022-11-27T15:00:33.239 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:33.239 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T15:00:33.239 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:33.240 INFO:teuthology.orchestra.run.smithi133.stdout:Install 2 Packages 2022-11-27T15:00:33.240 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:33.240 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 655 k 2022-11-27T15:00:33.240 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 1.8 M 2022-11-27T15:00:33.240 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T15:00:33.296 INFO:teuthology.orchestra.run.smithi133.stdout:(1/2): ceph-fuse-15.2.17-0.el8.x86_64.rpm 9.5 MB/s | 572 kB 00:00 2022-11-27T15:00:33.322 INFO:teuthology.orchestra.run.smithi133.stdout:(2/2): fuse-2.9.7-16.el8.x86_64.rpm 986 kB/s | 83 kB 00:00 2022-11-27T15:00:33.322 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:33.323 INFO:teuthology.orchestra.run.smithi133.stdout:Total 7.4 MB/s | 655 kB 00:00 2022-11-27T15:00:33.325 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:33.326 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:33.327 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:33.327 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:33.327 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2022-11-27T15:00:33.328 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:15.2.17-0.el8 ceph 572 k 2022-11-27T15:00:33.328 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2022-11-27T15:00:33.328 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2022-11-27T15:00:33.328 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:33.329 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T15:00:33.329 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:33.329 INFO:teuthology.orchestra.run.smithi102.stdout:Install 2 Packages 2022-11-27T15:00:33.329 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:33.330 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 655 k 2022-11-27T15:00:33.330 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 1.8 M 2022-11-27T15:00:33.330 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T15:00:33.332 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T15:00:33.353 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T15:00:33.353 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T15:00:33.430 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T15:00:33.433 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T15:00:33.570 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:33.571 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:33.571 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2022-11-27T15:00:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:33.572 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:33.572 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:33.631 INFO:teuthology.orchestra.run.smithi102.stdout:(1/2): ceph-fuse-15.2.17-0.el8.x86_64.rpm 1.8 MB/s | 572 kB 00:00 2022-11-27T15:00:33.665 INFO:teuthology.orchestra.run.smithi102.stdout:(2/2): fuse-2.9.7-16.el8.x86_64.rpm 246 kB/s | 83 kB 00:00 2022-11-27T15:00:33.667 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:33.667 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.9 MB/s | 655 kB 00:00 2022-11-27T15:00:33.676 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T15:00:33.697 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T15:00:33.697 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T15:00:33.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr 2022-11-27T15:00:33.765 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T15:00:33.768 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T15:00:33.915 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T15:00:33.942 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T15:00:34.134 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2022-11-27T15:00:34.182 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2022-11-27T15:00:34.277 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T15:00:34.341 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T15:00:34.382 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:08 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:34.634 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T15:00:34.635 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:15.2.17-0.el8.x86_64 1/2 2022-11-27T15:00:34.696 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T15:00:34.697 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:15.2.17-0.el8.x86_64 1/2 2022-11-27T15:00:35.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2022-11-27T15:00:35.049 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:35.049 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2022-11-27T15:00:35.050 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:15.2.17-0.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2022-11-27T15:00:35.050 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:35.050 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:35.149 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2022-11-27T15:00:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:35.150 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2022-11-27T15:00:35.150 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:15.2.17-0.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2022-11-27T15:00:35.151 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:35.151 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:35.265 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install librados-devel 2022-11-27T15:00:35.382 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados-devel 2022-11-27T15:00:35.450 INFO:teuthology.orchestra.run.smithi039.stdout:Package ceph-mgr-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:35.552 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:35.553 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T15:00:35.553 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:35.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-fuse 2022-11-27T15:00:35.871 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:11 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:35.989 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:11 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:36.264 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:10 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:36.937 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:36.939 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:36.939 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:36.939 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:36.940 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2022-11-27T15:00:36.940 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T15:00:36.940 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:36.940 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T15:00:36.941 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:36.941 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2022-11-27T15:00:36.941 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:36.941 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 140 k 2022-11-27T15:00:36.942 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 478 k 2022-11-27T15:00:36.942 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T15:00:37.001 INFO:teuthology.orchestra.run.smithi133.stdout:librados-devel-15.2.17-0.el8.x86_64.rpm 2.6 MB/s | 140 kB 00:00 2022-11-27T15:00:37.002 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:37.002 INFO:teuthology.orchestra.run.smithi133.stdout:Total 2.5 MB/s | 140 kB 00:00 2022-11-27T15:00:37.003 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T15:00:37.003 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T15:00:37.004 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T15:00:37.044 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T15:00:37.047 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T15:00:37.065 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:37.067 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:37.067 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:37.067 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:37.067 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2022-11-27T15:00:37.068 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T15:00:37.068 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:37.068 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T15:00:37.068 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:37.069 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2022-11-27T15:00:37.069 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:37.069 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 140 k 2022-11-27T15:00:37.069 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 478 k 2022-11-27T15:00:37.069 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T15:00:37.128 INFO:teuthology.orchestra.run.smithi102.stdout:librados-devel-15.2.17-0.el8.x86_64.rpm 2.3 MB/s | 140 kB 00:00 2022-11-27T15:00:37.129 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:37.129 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.2 MB/s | 140 kB 00:00 2022-11-27T15:00:37.130 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T15:00:37.134 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T15:00:37.134 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T15:00:37.149 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T15:00:37.180 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T15:00:37.183 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T15:00:37.306 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T15:00:37.310 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:37.422 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:37.423 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:37.424 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:37.424 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:37.424 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2022-11-27T15:00:37.424 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:15.2.17-0.el8 ceph 572 k 2022-11-27T15:00:37.425 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2022-11-27T15:00:37.425 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2022-11-27T15:00:37.425 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:37.425 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T15:00:37.426 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:37.426 INFO:teuthology.orchestra.run.smithi039.stdout:Install 2 Packages 2022-11-27T15:00:37.426 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:37.426 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 655 k 2022-11-27T15:00:37.426 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 1.8 M 2022-11-27T15:00:37.427 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T15:00:37.449 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:37.482 INFO:teuthology.orchestra.run.smithi039.stdout:(1/2): ceph-fuse-15.2.17-0.el8.x86_64.rpm 9.9 MB/s | 572 kB 00:00 2022-11-27T15:00:37.508 INFO:teuthology.orchestra.run.smithi039.stdout:(2/2): fuse-2.9.7-16.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2022-11-27T15:00:37.509 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:37.509 INFO:teuthology.orchestra.run.smithi039.stdout:Total 7.6 MB/s | 655 kB 00:00 2022-11-27T15:00:37.518 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T15:00:37.540 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T15:00:37.540 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T15:00:37.555 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:37.616 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T15:00:37.619 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T15:00:37.703 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:37.790 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T15:00:37.979 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:37.980 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:37.980 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2022-11-27T15:00:37.980 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:37.980 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:37.981 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:38.033 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2022-11-27T15:00:38.162 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:38.163 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:38.163 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2022-11-27T15:00:38.164 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:38.164 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:38.165 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:38.166 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install libcephfs2 2022-11-27T15:00:38.190 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T15:00:38.362 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs2 2022-11-27T15:00:38.558 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2022-11-27T15:00:38.559 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:15.2.17-0.el8.x86_64 1/2 2022-11-27T15:00:38.777 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:14 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:38.965 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:14 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:39.021 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2022-11-27T15:00:39.022 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:39.022 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2022-11-27T15:00:39.023 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:15.2.17-0.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2022-11-27T15:00:39.023 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:39.024 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:39.184 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install librados-devel 2022-11-27T15:00:39.765 INFO:teuthology.orchestra.run.smithi133.stdout:Package libcephfs2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:39.817 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:13 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:39.854 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:39.855 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T15:00:39.855 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:39.948 INFO:teuthology.orchestra.run.smithi102.stdout:Package libcephfs2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:40.038 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:40.040 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T15:00:40.040 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:40.292 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs-devel 2022-11-27T15:00:40.467 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install libcephfs-devel 2022-11-27T15:00:40.885 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:16 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:40.981 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:40.982 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:40.983 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:40.983 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2022-11-27T15:00:40.983 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T15:00:40.984 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T15:00:40.984 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2022-11-27T15:00:40.985 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 140 k 2022-11-27T15:00:40.985 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 478 k 2022-11-27T15:00:40.986 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T15:00:41.034 INFO:teuthology.orchestra.run.smithi039.stdout:librados-devel-15.2.17-0.el8.x86_64.rpm 3.1 MB/s | 140 kB 00:00 2022-11-27T15:00:41.035 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:41.035 INFO:teuthology.orchestra.run.smithi039.stdout:Total 3.0 MB/s | 140 kB 00:00 2022-11-27T15:00:41.036 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T15:00:41.036 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T15:00:41.037 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T15:00:41.066 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:17 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:41.085 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T15:00:41.088 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T15:00:41.196 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T15:00:41.347 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:41.591 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:41.972 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:41.973 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:41.973 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:41.974 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:41.974 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2022-11-27T15:00:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:15.2.17-0.el8 ceph 22 k 2022-11-27T15:00:41.974 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:41.975 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T15:00:41.975 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:41.975 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2022-11-27T15:00:41.975 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:41.976 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 22 k 2022-11-27T15:00:41.976 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 78 k 2022-11-27T15:00:41.976 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T15:00:42.013 INFO:teuthology.orchestra.run.smithi102.stdout:libcephfs-devel-15.2.17-0.el8.x86_64.rpm 551 kB/s | 22 kB 00:00 2022-11-27T15:00:42.014 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:42.015 INFO:teuthology.orchestra.run.smithi102.stdout:Total 535 kB/s | 22 kB 00:00 2022-11-27T15:00:42.015 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T15:00:42.018 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T15:00:42.018 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T15:00:42.019 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T15:00:42.022 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T15:00:42.024 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:42.025 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:42.025 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2022-11-27T15:00:42.026 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:42.026 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:42.074 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T15:00:42.149 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:42.150 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:42.150 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:42.151 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:42.151 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2022-11-27T15:00:42.151 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:15.2.17-0.el8 ceph 22 k 2022-11-27T15:00:42.152 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:42.152 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T15:00:42.152 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:42.152 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2022-11-27T15:00:42.152 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:42.153 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 22 k 2022-11-27T15:00:42.153 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 78 k 2022-11-27T15:00:42.153 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T15:00:42.212 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install libcephfs2 2022-11-27T15:00:42.227 INFO:teuthology.orchestra.run.smithi133.stdout:libcephfs-devel-15.2.17-0.el8.x86_64.rpm 314 kB/s | 22 kB 00:00 2022-11-27T15:00:42.228 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:42.228 INFO:teuthology.orchestra.run.smithi133.stdout:Total 308 kB/s | 22 kB 00:00 2022-11-27T15:00:42.228 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T15:00:42.229 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T15:00:42.229 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T15:00:42.229 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T15:00:42.230 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T15:00:42.248 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:42.276 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T15:00:42.458 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:42.459 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:42.668 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:42.839 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:42.839 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:42.839 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2022-11-27T15:00:42.840 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:42.840 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:42.840 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:42.844 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:16 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:42.956 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados2 2022-11-27T15:00:43.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:43.054 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:43.055 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2022-11-27T15:00:43.055 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:43.055 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:43.056 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:43.160 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install librados2 2022-11-27T15:00:43.558 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:19 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:43.757 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:19 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:Package libcephfs2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:44.025 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:44.026 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T15:00:44.027 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:44.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install libcephfs-devel 2022-11-27T15:00:44.534 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:44.626 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:44.627 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T15:00:44.627 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:44.722 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librbd1 2022-11-27T15:00:44.749 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:44.839 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:44.840 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T15:00:44.841 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:44.867 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:18 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:44.928 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install librbd1 2022-11-27T15:00:45.317 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:21 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:45.531 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:21 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:46.029 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:46.030 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:46.030 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:46.030 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:46.031 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2022-11-27T15:00:46.031 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:15.2.17-0.el8 ceph 22 k 2022-11-27T15:00:46.031 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:46.032 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T15:00:46.032 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:00:46.032 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2022-11-27T15:00:46.032 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:46.032 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 22 k 2022-11-27T15:00:46.033 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 78 k 2022-11-27T15:00:46.033 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T15:00:46.075 INFO:teuthology.orchestra.run.smithi039.stdout:libcephfs-devel-15.2.17-0.el8.x86_64.rpm 503 kB/s | 22 kB 00:00 2022-11-27T15:00:46.076 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:46.076 INFO:teuthology.orchestra.run.smithi039.stdout:Total 489 kB/s | 22 kB 00:00 2022-11-27T15:00:46.078 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T15:00:46.079 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T15:00:46.079 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T15:00:46.081 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T15:00:46.084 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T15:00:46.135 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T15:00:46.310 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:46.313 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:46.400 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:46.402 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T15:00:46.402 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:46.498 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rados 2022-11-27T15:00:46.534 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:46.550 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:46.625 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:46.626 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T15:00:46.627 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:46.716 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install python3-rados 2022-11-27T15:00:46.961 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:46.961 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:46.962 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2022-11-27T15:00:46.962 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:46.962 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:00:46.962 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:47.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install librados2 2022-11-27T15:00:47.093 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:23 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:47.315 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:23 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:47.706 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:21 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:48.032 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rados-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:48.124 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:48.125 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T15:00:48.125 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:48.220 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rgw 2022-11-27T15:00:48.310 INFO:teuthology.orchestra.run.smithi133.stdout:Package python3-rados-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:48.401 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:48.402 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T15:00:48.402 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:48.483 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install python3-rgw 2022-11-27T15:00:48.771 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:48.813 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:24 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:48.872 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:48.873 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T15:00:48.873 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:48.960 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install librbd1 2022-11-27T15:00:49.082 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:25 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:49.590 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:23 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:49.804 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rgw-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:49.896 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:49.897 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T15:00:49.898 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:49.991 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-cephfs 2022-11-27T15:00:50.071 INFO:teuthology.orchestra.run.smithi133.stdout:Package python3-rgw-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:50.163 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:50.164 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T15:00:50.165 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:50.244 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install python3-cephfs 2022-11-27T15:00:50.586 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:26 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:50.647 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:50.749 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:50.750 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T15:00:50.751 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:50.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-rados 2022-11-27T15:00:50.842 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:26 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:51.466 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:25 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:51.573 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-cephfs-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:51.663 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:51.664 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T15:00:51.665 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:51.758 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rbd 2022-11-27T15:00:51.835 INFO:teuthology.orchestra.run.smithi133.stdout:Package python3-cephfs-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:51.927 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:51.928 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T15:00:51.928 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:52.012 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install python3-rbd 2022-11-27T15:00:52.353 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:28 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:52.519 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-rados-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:52.612 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:28 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:52.619 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T15:00:52.621 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:52.704 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-rgw 2022-11-27T15:00:53.333 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:27 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:53.349 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rbd-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:53.441 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:53.442 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T15:00:53.442 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:53.540 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-fuse 2022-11-27T15:00:53.605 INFO:teuthology.orchestra.run.smithi133.stdout:Package python3-rbd-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:53.696 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:53.697 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T15:00:53.697 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:53.806 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install rbd-fuse 2022-11-27T15:00:54.133 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:30 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:54.388 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-rgw-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:54.413 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:30 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:54.490 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:54.492 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T15:00:54.492 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:54.711 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-cephfs 2022-11-27T15:00:55.248 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:55.249 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:55.250 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:55.250 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:55.250 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2022-11-27T15:00:55.250 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:15.2.17-0.el8 ceph 73 k 2022-11-27T15:00:55.251 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:55.251 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T15:00:55.251 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:55.252 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2022-11-27T15:00:55.252 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:55.253 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 73 k 2022-11-27T15:00:55.253 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 188 k 2022-11-27T15:00:55.254 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T15:00:55.293 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-fuse-15.2.17-0.el8.x86_64.rpm 1.6 MB/s | 73 kB 00:00 2022-11-27T15:00:55.295 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:55.296 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.6 MB/s | 73 kB 00:00 2022-11-27T15:00:55.296 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T15:00:55.303 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T15:00:55.304 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T15:00:55.341 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:29 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:55.349 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T15:00:55.352 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T15:00:55.445 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T15:00:55.498 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:55.499 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:55.499 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:55.500 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:55.500 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2022-11-27T15:00:55.500 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:15.2.17-0.el8 ceph 73 k 2022-11-27T15:00:55.500 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:55.500 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T15:00:55.501 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:55.501 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2022-11-27T15:00:55.501 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:55.501 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 73 k 2022-11-27T15:00:55.502 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 188 k 2022-11-27T15:00:55.502 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T15:00:55.556 INFO:teuthology.orchestra.run.smithi133.stdout:rbd-fuse-15.2.17-0.el8.x86_64.rpm 1.4 MB/s | 73 kB 00:00 2022-11-27T15:00:55.556 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:55.556 INFO:teuthology.orchestra.run.smithi133.stdout:Total 1.4 MB/s | 73 kB 00:00 2022-11-27T15:00:55.557 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T15:00:55.560 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T15:00:55.560 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T15:00:55.605 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T15:00:55.608 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T15:00:55.706 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T15:00:55.864 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:56.072 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:56.115 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:56.399 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-cephfs-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:56.494 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:56.512 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:56.512 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T15:00:56.512 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:56.585 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-rbd 2022-11-27T15:00:56.837 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:56.838 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:56.838 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2022-11-27T15:00:56.838 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:56.838 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:56.839 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:00:56.917 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:56.917 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:56.917 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2022-11-27T15:00:56.918 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:15.2.17-0.el8.x86_64 2022-11-27T15:00:56.918 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:56.918 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:00:57.038 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install rbd-mirror 2022-11-27T15:00:57.106 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-mirror 2022-11-27T15:00:57.221 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:31 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:57.649 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:33 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:57.715 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:33 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:00:58.286 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-rbd-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:00:58.387 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:00:58.389 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T15:00:58.389 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:00:58.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install rbd-fuse 2022-11-27T15:00:58.727 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:00:58.728 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:58.729 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:58.729 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2022-11-27T15:00:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:15.2.17-0.el8 ceph 2.7 M 2022-11-27T15:00:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:58.730 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T15:00:58.730 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:00:58.730 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2022-11-27T15:00:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:00:58.731 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 2.7 M 2022-11-27T15:00:58.731 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 9.8 M 2022-11-27T15:00:58.731 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T15:00:58.796 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:00:58.798 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:58.798 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2022-11-27T15:00:58.799 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:58.799 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2022-11-27T15:00:58.799 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:15.2.17-0.el8 ceph 2.7 M 2022-11-27T15:00:58.800 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:58.800 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T15:00:58.801 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:00:58.801 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2022-11-27T15:00:58.802 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:00:58.802 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 2.7 M 2022-11-27T15:00:58.803 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 9.8 M 2022-11-27T15:00:58.803 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T15:00:58.812 INFO:teuthology.orchestra.run.smithi133.stdout:rbd-mirror-15.2.17-0.el8.x86_64.rpm 33 MB/s | 2.7 MB 00:00 2022-11-27T15:00:58.813 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:58.813 INFO:teuthology.orchestra.run.smithi133.stdout:Total 33 MB/s | 2.7 MB 00:00 2022-11-27T15:00:58.814 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T15:00:58.824 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T15:00:58.824 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T15:00:58.871 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-mirror-15.2.17-0.el8.x86_64.rpm 37 MB/s | 2.7 MB 00:00 2022-11-27T15:00:58.872 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:00:58.873 INFO:teuthology.orchestra.run.smithi102.stdout:Total 36 MB/s | 2.7 MB 00:00 2022-11-27T15:00:58.873 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T15:00:58.879 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T15:00:58.882 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T15:00:58.882 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T15:00:58.883 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T15:00:58.939 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T15:00:58.942 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T15:00:59.097 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:33 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:00:59.226 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T15:00:59.284 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T15:00:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:59.449 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:59.822 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:00:59.831 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:00.235 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:00.236 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:00.236 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2022-11-27T15:01:00.236 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:15.2.17-0.el8.x86_64 2022-11-27T15:01:00.237 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:00.237 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:01:00.251 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:00.252 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:00.252 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2022-11-27T15:01:00.252 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:15.2.17-0.el8.x86_64 2022-11-27T15:01:00.252 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:00.253 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:01:00.267 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:01:00.269 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:01:00.269 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2022-11-27T15:01:00.270 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:01:00.270 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2022-11-27T15:01:00.270 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:15.2.17-0.el8 ceph 73 k 2022-11-27T15:01:00.271 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:00.271 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T15:01:00.271 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:01:00.271 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2022-11-27T15:01:00.272 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:00.272 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 73 k 2022-11-27T15:01:00.272 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 188 k 2022-11-27T15:01:00.272 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T15:01:00.321 INFO:teuthology.orchestra.run.smithi039.stdout:rbd-fuse-15.2.17-0.el8.x86_64.rpm 1.4 MB/s | 73 kB 00:00 2022-11-27T15:01:00.322 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:01:00.323 INFO:teuthology.orchestra.run.smithi039.stdout:Total 1.3 MB/s | 73 kB 00:00 2022-11-27T15:01:00.324 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T15:01:00.332 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T15:01:00.333 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T15:01:00.382 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T15:01:00.385 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T15:01:00.433 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install rbd-nbd 2022-11-27T15:01:00.476 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-nbd 2022-11-27T15:01:00.485 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T15:01:00.653 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:00.891 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:01.043 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:37 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:01:01.080 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:37 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:01:01.345 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:01.345 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:01.346 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2022-11-27T15:01:01.346 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:15.2.17-0.el8.x86_64 2022-11-27T15:01:01.346 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:01.346 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:01:01.539 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install rbd-mirror 2022-11-27T15:01:02.131 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:01:02.133 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:01:02.133 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2022-11-27T15:01:02.133 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:01:02.134 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2022-11-27T15:01:02.134 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T15:01:02.135 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:02.135 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2022-11-27T15:01:02.136 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2022-11-27T15:01:02.136 INFO:teuthology.orchestra.run.smithi133.stdout:Install 1 Package 2022-11-27T15:01:02.137 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:02.137 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 140 k 2022-11-27T15:01:02.138 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 418 k 2022-11-27T15:01:02.138 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2022-11-27T15:01:02.167 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:01:02.168 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:01:02.169 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2022-11-27T15:01:02.169 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:01:02.169 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2022-11-27T15:01:02.169 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T15:01:02.170 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:02.170 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2022-11-27T15:01:02.170 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2022-11-27T15:01:02.170 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2022-11-27T15:01:02.171 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:02.171 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 140 k 2022-11-27T15:01:02.171 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 418 k 2022-11-27T15:01:02.177 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2022-11-27T15:01:02.178 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:36 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:01:02.179 INFO:teuthology.orchestra.run.smithi133.stdout:rbd-nbd-15.2.17-0.el8.x86_64.rpm 3.5 MB/s | 140 kB 00:00 2022-11-27T15:01:02.180 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:01:02.180 INFO:teuthology.orchestra.run.smithi133.stdout:Total 3.4 MB/s | 140 kB 00:00 2022-11-27T15:01:02.181 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2022-11-27T15:01:02.190 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2022-11-27T15:01:02.190 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2022-11-27T15:01:02.237 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2022-11-27T15:01:02.237 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2022-11-27T15:01:02.241 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-nbd-15.2.17-0.el8.x86_64.rpm 1.9 MB/s | 140 kB 00:00 2022-11-27T15:01:02.242 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:01:02.242 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.9 MB/s | 140 kB 00:00 2022-11-27T15:01:02.242 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2022-11-27T15:01:02.252 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2022-11-27T15:01:02.253 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2022-11-27T15:01:02.303 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2022-11-27T15:01:02.304 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2022-11-27T15:01:02.338 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2022-11-27T15:01:02.403 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2022-11-27T15:01:02.521 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:02.579 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:02.772 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:02.812 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:03.182 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:03.183 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:03.183 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2022-11-27T15:01:03.183 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:15.2.17-0.el8.x86_64 2022-11-27T15:01:03.184 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:03.184 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:01:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:03.268 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:03.268 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2022-11-27T15:01:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:15.2.17-0.el8.x86_64 2022-11-27T15:01:03.269 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:03.269 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:01:03.336 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:01:03.337 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:01:03.337 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2022-11-27T15:01:03.338 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:01:03.338 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2022-11-27T15:01:03.338 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:15.2.17-0.el8 ceph 2.7 M 2022-11-27T15:01:03.339 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:03.339 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T15:01:03.339 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:01:03.339 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2022-11-27T15:01:03.340 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:03.340 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 2.7 M 2022-11-27T15:01:03.340 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 9.8 M 2022-11-27T15:01:03.341 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T15:01:03.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install librados2 2022-11-27T15:01:03.408 INFO:teuthology.orchestra.run.smithi039.stdout:rbd-mirror-15.2.17-0.el8.x86_64.rpm 39 MB/s | 2.7 MB 00:00 2022-11-27T15:01:03.409 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:01:03.410 INFO:teuthology.orchestra.run.smithi039.stdout:Total 38 MB/s | 2.7 MB 00:00 2022-11-27T15:01:03.410 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T15:01:03.421 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T15:01:03.422 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T15:01:03.452 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados2 2022-11-27T15:01:03.481 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T15:01:03.484 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T15:01:03.872 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T15:01:04.001 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:01:39 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:01:04.063 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:40 ago on Sun 27 Nov 2022 02:59:24 PM UTC. 2022-11-27T15:01:04.080 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:04.476 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:04.893 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:04.894 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2022-11-27T15:01:04.894 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:15.2.17-0.el8.x86_64 2022-11-27T15:01:04.895 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:04.895 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:01:04.998 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:01:05.050 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:01:05.089 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2022-11-27T15:01:05.090 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2022-11-27T15:01:05.091 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2022-11-27T15:01:05.093 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install rbd-nbd 2022-11-27T15:01:05.142 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2022-11-27T15:01:05.144 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2022-11-27T15:01:05.144 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2022-11-27T15:01:05.183 DEBUG:teuthology.parallel:result is None 2022-11-27T15:01:05.253 DEBUG:teuthology.parallel:result is None 2022-11-27T15:01:05.733 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:39 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:01:06.893 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:01:06.894 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:01:06.894 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2022-11-27T15:01:06.894 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:01:06.894 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2022-11-27T15:01:06.895 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:15.2.17-0.el8 ceph 140 k 2022-11-27T15:01:06.895 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:06.895 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2022-11-27T15:01:06.896 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2022-11-27T15:01:06.896 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2022-11-27T15:01:06.896 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:06.896 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 140 k 2022-11-27T15:01:06.897 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 418 k 2022-11-27T15:01:06.897 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2022-11-27T15:01:06.937 INFO:teuthology.orchestra.run.smithi039.stdout:rbd-nbd-15.2.17-0.el8.x86_64.rpm 3.2 MB/s | 140 kB 00:00 2022-11-27T15:01:06.938 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:01:06.938 INFO:teuthology.orchestra.run.smithi039.stdout:Total 3.1 MB/s | 140 kB 00:00 2022-11-27T15:01:06.938 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2022-11-27T15:01:06.949 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2022-11-27T15:01:06.949 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2022-11-27T15:01:06.999 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2022-11-27T15:01:07.002 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2022-11-27T15:01:07.108 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2022-11-27T15:01:07.285 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:07.526 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:07.958 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2022-11-27T15:01:07.959 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:07.959 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2022-11-27T15:01:07.959 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:15.2.17-0.el8.x86_64 2022-11-27T15:01:07.960 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:07.960 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:01:08.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install librados2 2022-11-27T15:01:08.758 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:42 ago on Sun 27 Nov 2022 02:59:26 PM UTC. 2022-11-27T15:01:09.817 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2022-11-27T15:01:09.918 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2022-11-27T15:01:09.919 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2022-11-27T15:01:09.919 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2022-11-27T15:01:09.998 DEBUG:teuthology.parallel:result is None 2022-11-27T15:01:09.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo lsb_release -is 2022-11-27T15:01:10.036 INFO:teuthology.orchestra.run.smithi039.stdout:CentOSStream 2022-11-27T15:01:10.037 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T15:01:10.038 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T15:01:10.038 INFO:teuthology.packaging:ref: None 2022-11-27T15:01:10.039 INFO:teuthology.packaging:tag: None 2022-11-27T15:01:10.039 INFO:teuthology.packaging:branch: octopus 2022-11-27T15:01:10.039 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T15:01:10.040 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T15:01:10.263 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T15:01:10.286 INFO:teuthology.orchestra.run.smithi039.stdout:15.2.17-0.el8 2022-11-27T15:01:10.287 INFO:teuthology.packaging:The installed version of ceph is 15.2.17-0.el8 2022-11-27T15:01:10.287 INFO:teuthology.task.install:The correct ceph version 15.2.17-0 is installed. 2022-11-27T15:01:10.288 DEBUG:teuthology.orchestra.run.smithi102:> sudo lsb_release -is 2022-11-27T15:01:10.331 INFO:teuthology.orchestra.run.smithi102.stdout:CentOSStream 2022-11-27T15:01:10.333 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T15:01:10.334 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T15:01:10.335 INFO:teuthology.packaging:ref: None 2022-11-27T15:01:10.336 INFO:teuthology.packaging:tag: None 2022-11-27T15:01:10.337 INFO:teuthology.packaging:branch: octopus 2022-11-27T15:01:10.337 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T15:01:10.338 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T15:01:10.570 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T15:01:10.594 INFO:teuthology.orchestra.run.smithi102.stdout:15.2.17-0.el8 2022-11-27T15:01:10.595 INFO:teuthology.packaging:The installed version of ceph is 15.2.17-0.el8 2022-11-27T15:01:10.595 INFO:teuthology.task.install:The correct ceph version 15.2.17-0 is installed. 2022-11-27T15:01:10.596 DEBUG:teuthology.orchestra.run.smithi133:> sudo lsb_release -is 2022-11-27T15:01:10.640 INFO:teuthology.orchestra.run.smithi133.stdout:CentOSStream 2022-11-27T15:01:10.642 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T15:01:10.642 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T15:01:10.642 INFO:teuthology.packaging:ref: None 2022-11-27T15:01:10.643 INFO:teuthology.packaging:tag: None 2022-11-27T15:01:10.643 INFO:teuthology.packaging:branch: octopus 2022-11-27T15:01:10.644 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T15:01:10.644 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2022-11-27T15:01:10.866 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T15:01:10.893 INFO:teuthology.orchestra.run.smithi133.stdout:15.2.17-0.el8 2022-11-27T15:01:10.893 INFO:teuthology.packaging:The installed version of ceph is 15.2.17-0.el8 2022-11-27T15:01:10.894 INFO:teuthology.task.install:The correct ceph version 15.2.17-0 is installed. 2022-11-27T15:01:10.894 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-11-27T15:01:10.895 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:10.895 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T15:01:10.925 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2022-11-27T15:01:10.925 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T15:01:10.958 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2022-11-27T15:01:10.959 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T15:01:10.992 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-11-27T15:01:10.993 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:10.993 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T15:01:11.028 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T15:01:11.097 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2022-11-27T15:01:11.098 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T15:01:11.127 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T15:01:11.196 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2022-11-27T15:01:11.197 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T15:01:11.230 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T15:01:11.305 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-11-27T15:01:11.306 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:11.307 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T15:01:11.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T15:01:11.403 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2022-11-27T15:01:11.403 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T15:01:11.432 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T15:01:11.501 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2022-11-27T15:01:11.502 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T15:01:11.533 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T15:01:11.604 INFO:teuthology.run_tasks:Running task print... 2022-11-27T15:01:11.617 INFO:teuthology.task.print:**** done installing octopus 2022-11-27T15:01:11.618 INFO:teuthology.run_tasks:Running task ceph... 2022-11-27T15:01:11.688 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-11-27T15:01:11.689 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /var/log/ceph 2022-11-27T15:01:11.692 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /var/log/ceph 2022-11-27T15:01:11.694 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /var/log/ceph 2022-11-27T15:01:11.724 INFO:tasks.ceph:Disabling ceph logrotate... 2022-11-27T15:01:11.725 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T15:01:11.761 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T15:01:11.764 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T15:01:11.792 INFO:tasks.ceph:Creating extra log directories... 2022-11-27T15:01:11.792 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T15:01:11.830 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T15:01:11.832 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T15:01:11.870 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-11-27T15:01:11.871 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-11-27T15:01:11.871 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088588', 'branch': 'main', 'description': 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '7088588', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'install ceph/octopus latest\n'}], 'name': 'yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'max_mds': 2}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['client.0'], ['client.1']], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'targets': {'smithi039.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNwrUDKayEO2S7Mie7vNpN3SFJxfD23cC8vb20TEAN5NN7zsq40dzi+Wy3JHohppj7rs7k1UoBsDyY7SFs+Ty19rJahL3b/toR2dUu+5jTzWaAmcTe4qfQXj5nPodGX7K57+oF9+jV8MO2f1yeFt8AV0wOhCV+6xH9SJNKI9ZR/WtkAUjJHFNj4z9/BgxBBCOnFGUuorDm+N42s/sn2AJu1JQm0QVDEfACbcfgsVVeKaM8A91xhP2PBEL9LyHkpPLoIEcaJgLX2T/o9tY/I36lVEWVdneB3qtifL9Feo7J7QCQCVbLYQ4ZazzW6evTZk1ckF7B8GpDMdW6J5FlRrKckfAU/P18uTssiSsqUBwVURd0bYudF+hl0CCIBd79eZnt/2CvEUif+RuYtlpowTwISmQ8QKWV3xe4hpVJiF0/s/RAeYanf4h6g8h8+5LgDjwjwDRhrX/TctT1Lyuw7lQNfJxmPi482Qn7DXNq+gyNnH6Vwgw8r4UvxIi6eVnWYck=', 'smithi102.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuf9IbSu5GEtrGO8nP78s5VBQ6HsMZOC92ssnMH6UAkn+RuwoJmXrwpbAjsXw9D0VGQEQwGljiluO5vFF74FGfYhjjvE2yzlpaj86TxTG846xCEtySGvz566diD4Z+eM3Qg7CSz3FiFY8NNEGkEj3BJ84equmr66PrlqIi+oEcsSE8rnT7XHoJPmsI/ppjMgqZn9FIyhyRXKV0u0NPsKf00k/PZ1ZNeRJCfSn30nimMmoOBoDE0iv+Lh/Mk+Sy4seY7Z51XmtSNrxmMMfkVwkygxDYzDR1WLeF/GvUBYLIGkkDYIWfu8yLGXxHzcNCH03TFYENdNbaSE87bV6nkRbFvrs4d+E4ySaddkKFdpwmkm+Lfkoz/6biykIIuEQRxXlMkcfImkqmpAJeUTkYTPpQ1R0xIx4ZB0ILeQfip0tVZq3FGnScGjWWDYdfmtW8NJ9zzEiNJElZKpqwFKiE8sNADtoWNOT1DKX+rP3JKRlm0eu/Wy8MDBadSLOB6Q8b+lM=', 'smithi133.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4Z11Kkjqy08SJTm/famI+TGALhySgEe4RuwwvwwoRSdcliFfIepI19Zrf3QVEGwPrsmBz08b+8LZFoTE/VM84JHBNWvxGnjattIadpv+80FJNJW3eFu9agc5apKpe2GGECuPZCZSXBgZ7IUhJQhxGySjYZ03KRqye+wi6sp3q13kw+IBWDX95ucyP+KCrnVeTovSbeCFiOuSXg+MKaDHBsI1LGCmiMVIFBHnd029kJTtIU4gtkUBggEBCjuwZbj8a7B9lmOh+zYP/xYc1oV0rZ+EDUI1Lmv4y6cEAi7k8S3aLbu5iWgypKYHNXdONxxLvcNpdLwYa+4JE4zb60zrYh43PqpuprqgD+7XYs+pIpPUS2zPjFW2fzJ+rAJKdOtEXZapk5jL/Ba/DAjUxLtSuvBW7x50tJm9xBPy7slL557INBDZLnQnSAZRfY1ueLW+oNflLJu5uuj5poCZt6KQrKUUky9HjNCiknfkBpSBW52z1UqSsRMAluztFZeI0ngE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'octopus', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, {'print': '**** done installing octopus'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cephfs': {'max_mds': 2}, 'flavor': 'default', 'fs': 'xfs', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client octopus']}}, {'print': '**** done ceph'}, {'ceph-fuse': None}, {'print': '**** done octopus client'}, {'workunit': {'clients': {'all': ['suites/fsstress.sh']}}}, {'print': '**** done fsstress'}, {'mds_pre_upgrade': None}, {'print': '**** done mds pre-upgrade sequence'}, {'install.upgrade': {'mon.a': {'branch': 'quincy'}, 'mon.b': {'branch': 'quincy'}}}, {'print': '**** done install.upgrade both hosts'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'ceph.stop': ['mds.*']}, {'ceph.restart': {'daemons': ['mds.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph osd dump -f json-pretty', 'ceph versions', 'ceph osd require-osd-release quincy', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}, {'workunit': {'clients': {'all': ['suites/fsstress.sh']}}}, {'print': '**** done fsstress'}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-23_17:24:26', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.11998'} 2022-11-27T15:01:11.872 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T15:01:11.899 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T15:01:11.903 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T15:01:11.921 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T15:01:11.961 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T15:01:11.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T15:01:11.992 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:11.993 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T15:01:12.045 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T15:01:12.046 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2022-11-27T15:01:12.103 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T15:01:12.104 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:01:12.104 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 376416 Links: 1 2022-11-27T15:01:12.104 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:01:12.104 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:01:12.105 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2022-11-27 15:01:04.257437780 +0000 2022-11-27T15:01:12.105 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2022-11-27 14:58:34.477024228 +0000 2022-11-27T15:01:12.105 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2022-11-27 14:58:34.477024228 +0000 2022-11-27T15:01:12.106 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2022-11-27T15:01:12.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T15:01:12.171 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2022-11-27T15:01:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2022-11-27T15:01:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000343711 s, 1.5 MB/s 2022-11-27T15:01:12.174 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T15:01:12.234 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2022-11-27T15:01:12.292 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T15:01:12.293 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:01:12.293 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 374423 Links: 1 2022-11-27T15:01:12.294 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:01:12.294 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:01:12.294 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2022-11-27 15:01:04.257437780 +0000 2022-11-27T15:01:12.294 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2022-11-27 14:58:35.031013277 +0000 2022-11-27T15:01:12.294 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2022-11-27 14:58:35.031013277 +0000 2022-11-27T15:01:12.295 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2022-11-27T15:01:12.295 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T15:01:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2022-11-27T15:01:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2022-11-27T15:01:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000382671 s, 1.3 MB/s 2022-11-27T15:01:12.362 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T15:01:12.422 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2022-11-27T15:01:12.481 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T15:01:12.482 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:01:12.482 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 380495 Links: 1 2022-11-27T15:01:12.483 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:01:12.483 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:01:12.483 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2022-11-27 15:01:04.258437760 +0000 2022-11-27T15:01:12.484 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2022-11-27 14:58:35.612001793 +0000 2022-11-27T15:01:12.484 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2022-11-27 14:58:35.612001793 +0000 2022-11-27T15:01:12.484 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2022-11-27T15:01:12.485 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T15:01:12.547 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2022-11-27T15:01:12.548 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2022-11-27T15:01:12.548 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000294138 s, 1.7 MB/s 2022-11-27T15:01:12.550 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T15:01:12.609 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2022-11-27T15:01:12.667 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T15:01:12.668 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T15:01:12.668 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 382421 Links: 1 2022-11-27T15:01:12.669 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T15:01:12.669 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T15:01:12.670 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2022-11-27 15:01:04.258437760 +0000 2022-11-27T15:01:12.670 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2022-11-27 14:58:36.156991020 +0000 2022-11-27T15:01:12.671 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2022-11-27 14:58:36.156991020 +0000 2022-11-27T15:01:12.671 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: - 2022-11-27T15:01:12.672 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T15:01:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2022-11-27T15:01:12.735 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2022-11-27T15:01:12.735 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000268834 s, 1.9 MB/s 2022-11-27T15:01:12.737 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T15:01:12.796 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T15:01:12.797 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2022-11-27T15:01:12.797 INFO:tasks.ceph:Generating config... 2022-11-27T15:01:12.799 INFO:tasks.ceph:[global] mon warn on pool no app = False 2022-11-27T15:01:12.799 INFO:tasks.ceph:[global] ms bind msgr2 = False 2022-11-27T15:01:12.799 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2022-11-27T15:01:12.800 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2022-11-27T15:01:12.800 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2022-11-27T15:01:12.800 INFO:tasks.ceph:[client] client mount timeout = 600 2022-11-27T15:01:12.800 INFO:tasks.ceph:[client] debug client = 20 2022-11-27T15:01:12.801 INFO:tasks.ceph:[client] debug ms = 1 2022-11-27T15:01:12.801 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-11-27T15:01:12.801 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-11-27T15:01:12.802 INFO:tasks.ceph:[mds] debug mds = 20 2022-11-27T15:01:12.802 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-11-27T15:01:12.802 INFO:tasks.ceph:[mds] debug ms = 1 2022-11-27T15:01:12.803 INFO:tasks.ceph:[mds] mds debug frag = True 2022-11-27T15:01:12.803 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-11-27T15:01:12.803 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-11-27T15:01:12.803 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-11-27T15:01:12.804 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-11-27T15:01:12.804 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-11-27T15:01:12.804 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-11-27T15:01:12.805 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-11-27T15:01:12.805 INFO:tasks.ceph:[mgr] debug ms = 1 2022-11-27T15:01:12.805 INFO:tasks.ceph:[mon] debug mon = 20 2022-11-27T15:01:12.806 INFO:tasks.ceph:[mon] debug ms = 1 2022-11-27T15:01:12.806 INFO:tasks.ceph:[mon] debug paxos = 20 2022-11-27T15:01:12.806 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-11-27T15:01:12.806 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2022-11-27T15:01:12.807 INFO:tasks.ceph:[osd] bdev async discard = True 2022-11-27T15:01:12.807 INFO:tasks.ceph:[osd] bdev enable discard = True 2022-11-27T15:01:12.807 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2022-11-27T15:01:12.808 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-11-27T15:01:12.808 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-11-27T15:01:12.808 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2022-11-27T15:01:12.808 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2022-11-27T15:01:12.809 INFO:tasks.ceph:[osd] debug ms = 1 2022-11-27T15:01:12.809 INFO:tasks.ceph:[osd] debug osd = 20 2022-11-27T15:01:12.809 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2022-11-27T15:01:12.810 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-11-27T15:01:12.810 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-11-27T15:01:12.810 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-11-27T15:01:12.811 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-11-27T15:01:12.811 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-11-27T15:01:12.811 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-11-27T15:01:12.812 INFO:tasks.ceph:Setting up mon.a... 2022-11-27T15:01:12.812 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-11-27T15:01:12.881 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.keyring 2022-11-27T15:01:12.884 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-11-27T15:01:12.965 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T15:01:13.033 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.39'), ('mon.b', '[v2:172.21.15.39:3301,v1:172.21.15.39:6790]'), ('mon.c', '[v2:172.21.15.39:3302,v1:172.21.15.39:6791]')] 2022-11-27T15:01:13.035 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],[v2:172.21.15.39:3302,v1:172.21.15.39:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds.c': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-11-27T15:01:13.036 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:13.037 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T15:01:13.091 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.39 --addv b '[v2:172.21.15.39:3301,v1:172.21.15.39:6790]' --addv c '[v2:172.21.15.39:3302,v1:172.21.15.39:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:01:13.176 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:01:13.177 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: generated fsid fb73c991-5ff5-4983-821e-38ecbf33964f 2022-11-27T15:01:13.177 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 0 2022-11-27T15:01:13.178 INFO:teuthology.orchestra.run.smithi039.stdout:fsid fb73c991-5ff5-4983-821e-38ecbf33964f 2022-11-27T15:01:13.178 INFO:teuthology.orchestra.run.smithi039.stdout:last_changed 2022-11-27T15:01:13.176069+0000 2022-11-27T15:01:13.178 INFO:teuthology.orchestra.run.smithi039.stdout:created 2022-11-27T15:01:13.176069+0000 2022-11-27T15:01:13.179 INFO:teuthology.orchestra.run.smithi039.stdout:min_mon_release 0 (unknown) 2022-11-27T15:01:13.179 INFO:teuthology.orchestra.run.smithi039.stdout:0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2022-11-27T15:01:13.179 INFO:teuthology.orchestra.run.smithi039.stdout:1: [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] mon.b 2022-11-27T15:01:13.179 INFO:teuthology.orchestra.run.smithi039.stdout:2: [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] mon.c 2022-11-27T15:01:13.179 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-11-27T15:01:13.181 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T15:01:13.199 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fb73c991-5ff5-4983-821e-38ecbf33964f... 2022-11-27T15:01:13.201 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T15:01:13.243 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T15:01:13.246 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T15:01:13.286 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2022-11-27T15:01:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2022-11-27T15:01:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T15:01:13.287 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2022-11-27T15:01:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2022-11-27T15:01:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2022-11-27T15:01:13.288 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2022-11-27T15:01:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2022-11-27T15:01:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.289 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2022-11-27T15:01:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2022-11-27T15:01:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T15:01:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2022-11-27T15:01:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T15:01:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T15:01:13.291 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2022-11-27T15:01:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2022-11-27T15:01:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2022-11-27T15:01:13.292 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2022-11-27T15:01:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2022-11-27T15:01:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2022-11-27T15:01:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2022-11-27T15:01:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2022-11-27T15:01:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T15:01:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T15:01:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2022-11-27T15:01:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.294 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T15:01:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2022-11-27T15:01:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.295 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2022-11-27T15:01:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2022-11-27T15:01:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2022-11-27T15:01:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2022-11-27T15:01:13.296 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],[v2:172.21.15.39:3302,v1:172.21.15.39:6791]" 2022-11-27T15:01:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on pool no app = False 2022-11-27T15:01:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: ms bind msgr2 = False 2022-11-27T15:01:13.297 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore warn on legacy statfs = False 2022-11-27T15:01:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore warn on no per pool omap = False 2022-11-27T15:01:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn min per osd = 0 2022-11-27T15:01:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = fb73c991-5ff5-4983-821e-38ecbf33964f 2022-11-27T15:01:13.298 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.298 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2022-11-27T15:01:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2022-11-27T15:01:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2022-11-27T15:01:13.299 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2022-11-27T15:01:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2022-11-27T15:01:13.300 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2022-11-27T15:01:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2022-11-27T15:01:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2022-11-27T15:01:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2022-11-27T15:01:13.301 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2022-11-27T15:01:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2022-11-27T15:01:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.302 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2022-11-27T15:01:13.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2022-11-27T15:01:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2022-11-27T15:01:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.304 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2022-11-27T15:01:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2022-11-27T15:01:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2022-11-27T15:01:13.305 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2022-11-27T15:01:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: bdev async discard = True 2022-11-27T15:01:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: bdev enable discard = True 2022-11-27T15:01:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore allocator = bitmap 2022-11-27T15:01:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2022-11-27T15:01:13.306 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2022-11-27T15:01:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 1/20 2022-11-27T15:01:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 1/20 2022-11-27T15:01:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2022-11-27T15:01:13.307 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2022-11-27T15:01:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 4/10 2022-11-27T15:01:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T15:01:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2022-11-27T15:01:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2022-11-27T15:01:13.308 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2022-11-27T15:01:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2022-11-27T15:01:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: osd op complaint time = 180 2022-11-27T15:01:13.309 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.309 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2022-11-27T15:01:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2022-11-27T15:01:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2022-11-27T15:01:13.310 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2022-11-27T15:01:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2022-11-27T15:01:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2022-11-27T15:01:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2022-11-27T15:01:13.311 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2022-11-27T15:01:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.312 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2022-11-27T15:01:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2022-11-27T15:01:13.312 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2022-11-27T15:01:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2022-11-27T15:01:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2022-11-27T15:01:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2022-11-27T15:01:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2022-11-27T15:01:13.313 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2022-11-27T15:01:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2022-11-27T15:01:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2022-11-27T15:01:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2022-11-27T15:01:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.314 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T15:01:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T15:01:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T15:01:13.315 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T15:01:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T15:01:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T15:01:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.322 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2022-11-27T15:01:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2022-11-27T15:01:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2022-11-27T15:01:13.323 INFO:teuthology.orchestra.run.smithi039.stdout: mon op complaint time = 120 2022-11-27T15:01:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = False 2022-11-27T15:01:13.324 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:13.325 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2022-11-27T15:01:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2022-11-27T15:01:13.325 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2022-11-27T15:01:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2022-11-27T15:01:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T15:01:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T15:01:13.327 INFO:teuthology.orchestra.run.smithi039.stdout: client mount timeout = 600 2022-11-27T15:01:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: debug client = 20 2022-11-27T15:01:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2022-11-27T15:01:13.328 INFO:teuthology.orchestra.run.smithi039.stdout: rados mon op timeout = 900 2022-11-27T15:01:13.329 INFO:teuthology.orchestra.run.smithi039.stdout: rados osd op timeout = 900 2022-11-27T15:01:13.329 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2022-11-27T15:01:13.330 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2022-11-27T15:01:13.330 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2022-11-27T15:01:13.331 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2022-11-27T15:01:13.331 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2022-11-27T15:01:13.331 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2022-11-27T15:01:13.332 INFO:teuthology.orchestra.run.smithi039.stdout:[mds] 2022-11-27T15:01:13.332 INFO:teuthology.orchestra.run.smithi039.stdout: debug mds = 20 2022-11-27T15:01:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: debug mds balancer = 20 2022-11-27T15:01:13.333 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2022-11-27T15:01:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: mds debug frag = True 2022-11-27T15:01:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: mds debug scatterstat = True 2022-11-27T15:01:13.334 INFO:teuthology.orchestra.run.smithi039.stdout: mds op complaint time = 180 2022-11-27T15:01:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: mds verify scatter = True 2022-11-27T15:01:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: osd op complaint time = 180 2022-11-27T15:01:13.335 INFO:teuthology.orchestra.run.smithi039.stdout: rados mon op timeout = 900 2022-11-27T15:01:13.336 INFO:teuthology.orchestra.run.smithi039.stdout: rados osd op timeout = 900 2022-11-27T15:01:13.339 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2022-11-27T15:01:13.339 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2022-11-27T15:01:13.339 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T15:01:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2022-11-27T15:01:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2022-11-27T15:01:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2022-11-27T15:01:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2022-11-27T15:01:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2022-11-27T15:01:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.342 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2022-11-27T15:01:13.342 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2022-11-27T15:01:13.342 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.343 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T15:01:13.343 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2022-11-27T15:01:13.344 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.344 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T15:01:13.345 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T15:01:13.345 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.345 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2022-11-27T15:01:13.346 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.346 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2022-11-27T15:01:13.347 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2022-11-27T15:01:13.347 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2022-11-27T15:01:13.347 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2022-11-27T15:01:13.348 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2022-11-27T15:01:13.348 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2022-11-27T15:01:13.349 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2022-11-27T15:01:13.349 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T15:01:13.350 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T15:01:13.350 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2022-11-27T15:01:13.350 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.351 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T15:01:13.351 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.352 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2022-11-27T15:01:13.352 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.352 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2022-11-27T15:01:13.353 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.353 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2022-11-27T15:01:13.354 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2022-11-27T15:01:13.354 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2022-11-27T15:01:13.354 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],[v2:172.21.15.39:3302,v1:172.21.15.39:6791]" 2022-11-27T15:01:13.355 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on pool no app = False 2022-11-27T15:01:13.355 INFO:teuthology.orchestra.run.smithi133.stdout: ms bind msgr2 = False 2022-11-27T15:01:13.356 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore warn on legacy statfs = False 2022-11-27T15:01:13.356 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore warn on no per pool omap = False 2022-11-27T15:01:13.357 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn min per osd = 0 2022-11-27T15:01:13.357 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = fb73c991-5ff5-4983-821e-38ecbf33964f 2022-11-27T15:01:13.357 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.358 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2022-11-27T15:01:13.358 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2022-11-27T15:01:13.358 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.359 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2022-11-27T15:01:13.359 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2022-11-27T15:01:13.360 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.360 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2022-11-27T15:01:13.360 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2022-11-27T15:01:13.361 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.361 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2022-11-27T15:01:13.362 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2022-11-27T15:01:13.362 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2022-11-27T15:01:13.363 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.363 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2022-11-27T15:01:13.364 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2022-11-27T15:01:13.364 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.365 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2022-11-27T15:01:13.365 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.365 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2022-11-27T15:01:13.366 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.366 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2022-11-27T15:01:13.367 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.367 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2022-11-27T15:01:13.368 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2022-11-27T15:01:13.368 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.368 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2022-11-27T15:01:13.369 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2022-11-27T15:01:13.369 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2022-11-27T15:01:13.369 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2022-11-27T15:01:13.369 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2022-11-27T15:01:13.370 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2022-11-27T15:01:13.370 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2022-11-27T15:01:13.370 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2022-11-27T15:01:13.370 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2022-11-27T15:01:13.370 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2022-11-27T15:01:13.371 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2022-11-27T15:01:13.371 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2022-11-27T15:01:13.371 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T15:01:13.371 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2022-11-27T15:01:13.372 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2022-11-27T15:01:13.372 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2022-11-27T15:01:13.372 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2022-11-27T15:01:13.372 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2022-11-27T15:01:13.372 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.373 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2022-11-27T15:01:13.373 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2022-11-27T15:01:13.373 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2022-11-27T15:01:13.373 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2022-11-27T15:01:13.373 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2022-11-27T15:01:13.374 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2022-11-27T15:01:13.374 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2022-11-27T15:01:13.374 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2022-11-27T15:01:13.374 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.375 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2022-11-27T15:01:13.375 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2022-11-27T15:01:13.375 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2022-11-27T15:01:13.375 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2022-11-27T15:01:13.376 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2022-11-27T15:01:13.376 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2022-11-27T15:01:13.377 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2022-11-27T15:01:13.377 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2022-11-27T15:01:13.377 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2022-11-27T15:01:13.378 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2022-11-27T15:01:13.378 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2022-11-27T15:01:13.379 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.379 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T15:01:13.380 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T15:01:13.380 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T15:01:13.380 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.380 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T15:01:13.381 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T15:01:13.381 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T15:01:13.381 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.381 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2022-11-27T15:01:13.381 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2022-11-27T15:01:13.382 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.382 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2022-11-27T15:01:13.382 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2022-11-27T15:01:13.382 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = False 2022-11-27T15:01:13.383 INFO:teuthology.orchestra.run.smithi133.stdout: 2022-11-27T15:01:13.383 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2022-11-27T15:01:13.383 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2022-11-27T15:01:13.383 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2022-11-27T15:01:13.383 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2022-11-27T15:01:13.384 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T15:01:13.384 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T15:01:13.384 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2022-11-27T15:01:13.384 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2022-11-27T15:01:13.384 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2022-11-27T15:01:13.385 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2022-11-27T15:01:13.385 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2022-11-27T15:01:13.385 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2022-11-27T15:01:13.385 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2022-11-27T15:01:13.385 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2022-11-27T15:01:13.386 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2022-11-27T15:01:13.386 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2022-11-27T15:01:13.386 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2022-11-27T15:01:13.386 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2022-11-27T15:01:13.387 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2022-11-27T15:01:13.387 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2022-11-27T15:01:13.387 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2022-11-27T15:01:13.387 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2022-11-27T15:01:13.387 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = True 2022-11-27T15:01:13.388 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2022-11-27T15:01:13.388 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = True 2022-11-27T15:01:13.388 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2022-11-27T15:01:13.388 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2022-11-27T15:01:13.388 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2022-11-27T15:01:13.389 INFO:teuthology.orchestra.run.smithi102.stdout:[global] 2022-11-27T15:01:13.389 INFO:teuthology.orchestra.run.smithi102.stdout: chdir = "" 2022-11-27T15:01:13.389 INFO:teuthology.orchestra.run.smithi102.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T15:01:13.389 INFO:teuthology.orchestra.run.smithi102.stdout: auth supported = cephx 2022-11-27T15:01:13.390 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.390 INFO:teuthology.orchestra.run.smithi102.stdout: filestore xattr use omap = true 2022-11-27T15:01:13.390 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.390 INFO:teuthology.orchestra.run.smithi102.stdout: mon clock drift allowed = 1.000 2022-11-27T15:01:13.390 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.391 INFO:teuthology.orchestra.run.smithi102.stdout: osd crush chooseleaf type = 0 2022-11-27T15:01:13.391 INFO:teuthology.orchestra.run.smithi102.stdout: auth debug = true 2022-11-27T15:01:13.391 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.391 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on old message = true 2022-11-27T15:01:13.392 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on bug = true 2022-11-27T15:01:13.392 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.392 INFO:teuthology.orchestra.run.smithi102.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T15:01:13.392 INFO:teuthology.orchestra.run.smithi102.stdout: mon pg warn max object skew = 0 2022-11-27T15:01:13.392 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.393 INFO:teuthology.orchestra.run.smithi102.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T15:01:13.393 INFO:teuthology.orchestra.run.smithi102.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T15:01:13.393 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.393 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default size = 2 2022-11-27T15:01:13.394 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.394 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow primary affinity = true 2022-11-27T15:01:13.394 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow pg remap = true 2022-11-27T15:01:13.394 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on legacy crush tunables = false 2022-11-27T15:01:13.394 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on crush straw calc version zero = false 2022-11-27T15:01:13.395 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on no sortbitwise = false 2022-11-27T15:01:13.395 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on osd down out interval zero = false 2022-11-27T15:01:13.395 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on too few osds = false 2022-11-27T15:01:13.395 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T15:01:13.395 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T15:01:13.396 INFO:teuthology.orchestra.run.smithi102.stdout: mon_allow_pool_size_one = true 2022-11-27T15:01:13.396 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.396 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T15:01:13.396 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.396 INFO:teuthology.orchestra.run.smithi102.stdout: osd default data pool replay window = 5 2022-11-27T15:01:13.397 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.397 INFO:teuthology.orchestra.run.smithi102.stdout: mon allow pool delete = true 2022-11-27T15:01:13.397 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.397 INFO:teuthology.orchestra.run.smithi102.stdout: mon cluster log file level = debug 2022-11-27T15:01:13.398 INFO:teuthology.orchestra.run.smithi102.stdout: debug asserts on shutdown = true 2022-11-27T15:01:13.398 INFO:teuthology.orchestra.run.smithi102.stdout: mon health detail to clog = false 2022-11-27T15:01:13.398 INFO:teuthology.orchestra.run.smithi102.stdout: mon host = "172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],[v2:172.21.15.39:3302,v1:172.21.15.39:6791]" 2022-11-27T15:01:13.398 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on pool no app = False 2022-11-27T15:01:13.398 INFO:teuthology.orchestra.run.smithi102.stdout: ms bind msgr2 = False 2022-11-27T15:01:13.399 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore warn on legacy statfs = False 2022-11-27T15:01:13.399 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore warn on no per pool omap = False 2022-11-27T15:01:13.399 INFO:teuthology.orchestra.run.smithi102.stdout: mon pg warn min per osd = 0 2022-11-27T15:01:13.399 INFO:teuthology.orchestra.run.smithi102.stdout: fsid = fb73c991-5ff5-4983-821e-38ecbf33964f 2022-11-27T15:01:13.399 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.400 INFO:teuthology.orchestra.run.smithi102.stdout:[osd] 2022-11-27T15:01:13.400 INFO:teuthology.orchestra.run.smithi102.stdout: osd journal size = 100 2022-11-27T15:01:13.400 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.400 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub load threshold = 5.0 2022-11-27T15:01:13.401 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub max interval = 600 2022-11-27T15:01:13.401 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.401 INFO:teuthology.orchestra.run.smithi102.stdout: osd recover clone overlap = true 2022-11-27T15:01:13.401 INFO:teuthology.orchestra.run.smithi102.stdout: osd recovery max chunk = 1048576 2022-11-27T15:01:13.401 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.402 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug shutdown = true 2022-11-27T15:01:13.402 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug op order = true 2022-11-27T15:01:13.402 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug verify stray on activate = true 2022-11-27T15:01:13.402 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.402 INFO:teuthology.orchestra.run.smithi102.stdout: osd open classes on start = true 2022-11-27T15:01:13.403 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug pg log writeout = true 2022-11-27T15:01:13.403 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.403 INFO:teuthology.orchestra.run.smithi102.stdout: osd deep scrub update digest min age = 30 2022-11-27T15:01:13.403 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.403 INFO:teuthology.orchestra.run.smithi102.stdout: osd map max advance = 10 2022-11-27T15:01:13.404 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.404 INFO:teuthology.orchestra.run.smithi102.stdout: journal zero on create = true 2022-11-27T15:01:13.404 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.404 INFO:teuthology.orchestra.run.smithi102.stdout: filestore ondisk finisher threads = 3 2022-11-27T15:01:13.405 INFO:teuthology.orchestra.run.smithi102.stdout: filestore apply finisher threads = 3 2022-11-27T15:01:13.405 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.405 INFO:teuthology.orchestra.run.smithi102.stdout: bdev debug aio = true 2022-11-27T15:01:13.405 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug misdirected ops = true 2022-11-27T15:01:13.405 INFO:teuthology.orchestra.run.smithi102.stdout: bdev async discard = True 2022-11-27T15:01:13.406 INFO:teuthology.orchestra.run.smithi102.stdout: bdev enable discard = True 2022-11-27T15:01:13.406 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore allocator = bitmap 2022-11-27T15:01:13.406 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore block size = 96636764160 2022-11-27T15:01:13.406 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore fsck on mount = True 2022-11-27T15:01:13.407 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluefs = 1/20 2022-11-27T15:01:13.407 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluestore = 1/20 2022-11-27T15:01:13.407 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2022-11-27T15:01:13.407 INFO:teuthology.orchestra.run.smithi102.stdout: debug osd = 20 2022-11-27T15:01:13.408 INFO:teuthology.orchestra.run.smithi102.stdout: debug rocksdb = 4/10 2022-11-27T15:01:13.408 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T15:01:13.409 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd full ratio = 0.9 2022-11-27T15:01:13.409 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd nearfull ratio = 0.8 2022-11-27T15:01:13.410 INFO:teuthology.orchestra.run.smithi102.stdout: osd failsafe full ratio = 0.95 2022-11-27T15:01:13.410 INFO:teuthology.orchestra.run.smithi102.stdout: osd objectstore = bluestore 2022-11-27T15:01:13.410 INFO:teuthology.orchestra.run.smithi102.stdout: osd op complaint time = 180 2022-11-27T15:01:13.411 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.411 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr] 2022-11-27T15:01:13.411 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2022-11-27T15:01:13.411 INFO:teuthology.orchestra.run.smithi102.stdout: debug mgr = 20 2022-11-27T15:01:13.412 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2022-11-27T15:01:13.412 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2022-11-27T15:01:13.412 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2022-11-27T15:01:13.412 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2022-11-27T15:01:13.412 INFO:teuthology.orchestra.run.smithi102.stdout: mgr/telemetry/nag = false 2022-11-27T15:01:13.413 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.413 INFO:teuthology.orchestra.run.smithi102.stdout:[mon] 2022-11-27T15:01:13.413 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2022-11-27T15:01:13.413 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2022-11-27T15:01:13.413 INFO:teuthology.orchestra.run.smithi102.stdout: debug paxos = 20 2022-11-27T15:01:13.414 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2022-11-27T15:01:13.414 INFO:teuthology.orchestra.run.smithi102.stdout: mon data avail warn = 5 2022-11-27T15:01:13.414 INFO:teuthology.orchestra.run.smithi102.stdout: mon mgr mkfs grace = 240 2022-11-27T15:01:13.414 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2022-11-27T15:01:13.415 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd reporter subtree level = osd 2022-11-27T15:01:13.415 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd prime pg temp = true 2022-11-27T15:01:13.415 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2022-11-27T15:01:13.415 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.415 INFO:teuthology.orchestra.run.smithi102.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T15:01:13.416 INFO:teuthology.orchestra.run.smithi102.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T15:01:13.416 INFO:teuthology.orchestra.run.smithi102.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T15:01:13.416 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.416 INFO:teuthology.orchestra.run.smithi102.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T15:01:13.416 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T15:01:13.417 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T15:01:13.417 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.417 INFO:teuthology.orchestra.run.smithi102.stdout: # 1m isn't quite enough 2022-11-27T15:01:13.417 INFO:teuthology.orchestra.run.smithi102.stdout: mon_down_mkfs_grace = 2m 2022-11-27T15:01:13.418 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.418 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_filestore_osds = false 2022-11-27T15:01:13.418 INFO:teuthology.orchestra.run.smithi102.stdout: mon op complaint time = 120 2022-11-27T15:01:13.418 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on osd down out interval zero = False 2022-11-27T15:01:13.418 INFO:teuthology.orchestra.run.smithi102.stdout: 2022-11-27T15:01:13.419 INFO:teuthology.orchestra.run.smithi102.stdout:[client] 2022-11-27T15:01:13.419 INFO:teuthology.orchestra.run.smithi102.stdout: rgw cache enabled = true 2022-11-27T15:01:13.419 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable ops log = true 2022-11-27T15:01:13.419 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable usage log = true 2022-11-27T15:01:13.420 INFO:teuthology.orchestra.run.smithi102.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T15:01:13.420 INFO:teuthology.orchestra.run.smithi102.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T15:01:13.420 INFO:teuthology.orchestra.run.smithi102.stdout: client mount timeout = 600 2022-11-27T15:01:13.420 INFO:teuthology.orchestra.run.smithi102.stdout: debug client = 20 2022-11-27T15:01:13.420 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2022-11-27T15:01:13.421 INFO:teuthology.orchestra.run.smithi102.stdout: rados mon op timeout = 900 2022-11-27T15:01:13.421 INFO:teuthology.orchestra.run.smithi102.stdout: rados osd op timeout = 900 2022-11-27T15:01:13.421 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.a] 2022-11-27T15:01:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.b] 2022-11-27T15:01:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.c] 2022-11-27T15:01:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.a] 2022-11-27T15:01:13.422 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.b] 2022-11-27T15:01:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:[mds.c] 2022-11-27T15:01:13.423 INFO:teuthology.orchestra.run.smithi102.stdout:[mds] 2022-11-27T15:01:13.423 INFO:teuthology.orchestra.run.smithi102.stdout: debug mds = 20 2022-11-27T15:01:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: debug mds balancer = 20 2022-11-27T15:01:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2022-11-27T15:01:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: mds debug frag = True 2022-11-27T15:01:13.424 INFO:teuthology.orchestra.run.smithi102.stdout: mds debug scatterstat = True 2022-11-27T15:01:13.425 INFO:teuthology.orchestra.run.smithi102.stdout: mds op complaint time = 180 2022-11-27T15:01:13.425 INFO:teuthology.orchestra.run.smithi102.stdout: mds verify scatter = True 2022-11-27T15:01:13.425 INFO:teuthology.orchestra.run.smithi102.stdout: osd op complaint time = 180 2022-11-27T15:01:13.425 INFO:teuthology.orchestra.run.smithi102.stdout: rados mon op timeout = 900 2022-11-27T15:01:13.425 INFO:teuthology.orchestra.run.smithi102.stdout: rados osd op timeout = 900 2022-11-27T15:01:13.426 INFO:tasks.ceph:Creating admin key on mon.a... 2022-11-27T15:01:13.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-11-27T15:01:13.480 INFO:tasks.ceph:Copying monmap to all nodes... 2022-11-27T15:01:13.480 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:13.481 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-11-27T15:01:13.499 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:13.500 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-11-27T15:01:13.557 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi039.front.sepia.ceph.com 2022-11-27T15:01:13.558 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:13.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T15:01:13.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T15:01:13.636 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:13.636 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:01:13.695 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi102.front.sepia.ceph.com 2022-11-27T15:01:13.696 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2022-11-27T15:01:13.696 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T15:01:13.697 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T15:01:13.733 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2022-11-27T15:01:13.734 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:01:13.792 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi133.front.sepia.ceph.com 2022-11-27T15:01:13.792 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2022-11-27T15:01:13.793 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T15:01:13.793 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T15:01:13.832 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2022-11-27T15:01:13.833 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:01:13.890 INFO:tasks.ceph:Setting up mon nodes... 2022-11-27T15:01:13.891 INFO:tasks.ceph:Setting up mgr nodes... 2022-11-27T15:01:13.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T15:01:13.942 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T15:01:13.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T15:01:13.996 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T15:01:13.999 INFO:tasks.ceph:Setting up mds nodes... 2022-11-27T15:01:14.000 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T15:01:14.053 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T15:01:14.054 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-11-27T15:01:14.082 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T15:01:14.172 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T15:01:14.174 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-11-27T15:01:14.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T15:01:14.292 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T15:01:14.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-11-27T15:01:14.323 INFO:tasks.ceph_client:Setting up client nodes... 2022-11-27T15:01:14.324 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-11-27T15:01:14.392 INFO:teuthology.orchestra.run.smithi102.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-11-27T15:01:14.406 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-11-27T15:01:14.465 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-11-27T15:01:14.477 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-11-27T15:01:14.477 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2022-11-27T15:01:14.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-11-27T15:01:14.506 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T15:01:14.507 INFO:tasks.ceph:role: osd.0 2022-11-27T15:01:14.507 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com 2022-11-27T15:01:14.507 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T15:01:14.602 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:01:14.603 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:01:14.603 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:01:14.603 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:01:14.604 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:01:14.604 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:01:14.604 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:01:14.605 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:01:14.605 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:01:14.606 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:01:15.895 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2022-11-27T15:01:15.897 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2022-11-27T15:01:15.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-11-27T15:01:15.943 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-11-27T15:01:15.978 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-11-27T15:01:16.047 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T15:01:16.048 INFO:tasks.ceph:role: osd.1 2022-11-27T15:01:16.049 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com 2022-11-27T15:01:16.049 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T15:01:16.144 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:01:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:01:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:01:16.144 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:01:16.145 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:01:16.145 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:01:16.145 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:01:16.145 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:01:16.146 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:01:16.146 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:01:17.441 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2022-11-27T15:01:17.443 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2022-11-27T15:01:17.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-11-27T15:01:17.490 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-11-27T15:01:17.525 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-11-27T15:01:17.593 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T15:01:17.594 INFO:tasks.ceph:role: osd.2 2022-11-27T15:01:17.594 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com 2022-11-27T15:01:17.595 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T15:01:17.688 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:01:17.689 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:01:17.689 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:01:17.689 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:01:17.690 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:01:17.690 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:01:17.690 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:01:17.690 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:01:17.690 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:01:17.691 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:01:18.975 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2022-11-27T15:01:18.977 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2022-11-27T15:01:18.978 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-11-27T15:01:19.030 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-11-27T15:01:19.068 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-11-27T15:01:19.137 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T15:01:19.138 INFO:tasks.ceph:role: osd.3 2022-11-27T15:01:19.138 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com 2022-11-27T15:01:19.138 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T15:01:19.234 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T15:01:19.235 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T15:01:19.235 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T15:01:19.235 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T15:01:19.235 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T15:01:19.236 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2022-11-27T15:01:19.236 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T15:01:19.236 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T15:01:19.236 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T15:01:19.237 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T15:01:20.517 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2022-11-27T15:01:20.519 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2022-11-27T15:01:20.520 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-11-27T15:01:20.588 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-11-27T15:01:20.623 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:01:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:20.707+0000 7f424da65f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-11-27T15:01:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:20.707+0000 7f424da65f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-11-27T15:01:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:20.707+0000 7f424da65f00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T15:01:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:20.707+0000 7f424da65f00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T15:01:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:20.707+0000 7f424da65f00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T15:01:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:20.708+0000 7f424da65f00 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-11-27T15:01:20.730 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:20.728+0000 7f424da65f00 -1 freelist read_size_meta_from_db missing size meta in DB 2022-11-27T15:01:23.054 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-11-27T15:01:23.084 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:01:23.170 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:23.169+0000 7f3013242f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-11-27T15:01:23.171 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:23.169+0000 7f3013242f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-11-27T15:01:23.171 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:23.169+0000 7f3013242f00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T15:01:23.171 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:23.169+0000 7f3013242f00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T15:01:23.171 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:23.169+0000 7f3013242f00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T15:01:23.172 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:23.170+0000 7f3013242f00 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-11-27T15:01:23.196 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:23.194+0000 7f3013242f00 -1 freelist read_size_meta_from_db missing size meta in DB 2022-11-27T15:01:25.519 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-11-27T15:01:25.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:01:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:25.638+0000 7f1e806ccf00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-11-27T15:01:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:25.638+0000 7f1e806ccf00 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-11-27T15:01:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:25.639+0000 7f1e806ccf00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T15:01:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:25.639+0000 7f1e806ccf00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T15:01:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:25.639+0000 7f1e806ccf00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T15:01:25.641 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:25.639+0000 7f1e806ccf00 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-11-27T15:01:25.654 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:25.654+0000 7f1e806ccf00 -1 freelist read_size_meta_from_db missing size meta in DB 2022-11-27T15:01:27.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-11-27T15:01:28.003 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:01:28.091 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:28.089+0000 7f5bcd460f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-11-27T15:01:28.091 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:28.089+0000 7f5bcd460f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-11-27T15:01:28.092 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:28.089+0000 7f5bcd460f00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T15:01:28.092 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:28.090+0000 7f5bcd460f00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T15:01:28.092 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:28.090+0000 7f5bcd460f00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T15:01:28.093 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:28.090+0000 7f5bcd460f00 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-11-27T15:01:28.117 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:28.115+0000 7f5bcd460f00 -1 freelist read_size_meta_from_db missing size meta in DB 2022-11-27T15:01:30.436 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-11-27T15:01:30.468 INFO:tasks.ceph:Reading keys from all nodes... 2022-11-27T15:01:30.469 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:30.470 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-11-27T15:01:30.537 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:30.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-11-27T15:01:30.605 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:30.606 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-11-27T15:01:30.675 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:30.675 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-11-27T15:01:30.743 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:30.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-11-27T15:01:30.811 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:30.812 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-11-27T15:01:30.879 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:30.879 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-11-27T15:01:30.947 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:30.947 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-11-27T15:01:31.015 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:31.015 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-11-27T15:01:31.086 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2022-11-27T15:01:31.086 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-11-27T15:01:31.107 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2022-11-27T15:01:31.108 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-11-27T15:01:31.130 INFO:tasks.ceph:Adding keys to all mons... 2022-11-27T15:01:31.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T15:01:31.156 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.x] 2022-11-27T15:01:31.157 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQC5e4NjylstOBAASe0ds4f1EtT2bcrRFlD7hg== 2022-11-27T15:01:31.157 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.y] 2022-11-27T15:01:31.158 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQC5e4NjPWdnOxAAWdiKs0vCVXydWgYdTpEHbA== 2022-11-27T15:01:31.159 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2022-11-27T15:01:31.159 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQC6e4NjltQIAxAAXWjzfRzJR3X96MNOIHXFIA== 2022-11-27T15:01:31.159 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2022-11-27T15:01:31.160 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQC6e4Njc7lHChAAdwZX+TJRktdpZoojbVNC7g== 2022-11-27T15:01:31.160 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2022-11-27T15:01:31.161 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQC6e4NjazFuERAAXYpmfvx6fpV9J06gW9RWVQ== 2022-11-27T15:01:31.161 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.0] 2022-11-27T15:01:31.161 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDAe4NjkRM5KhAAgT5eiornC7IOonJREXJiUQ== 2022-11-27T15:01:31.162 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.1] 2022-11-27T15:01:31.162 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDDe4Nj/PUYChAAYbJujd+9O+jkCXUA+BC9jw== 2022-11-27T15:01:31.162 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.2] 2022-11-27T15:01:31.162 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDFe4NjeecSJhAAfwC8nRulCl2lViD4zGredQ== 2022-11-27T15:01:31.163 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.3] 2022-11-27T15:01:31.163 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDIe4Nja9xoBRAABNWUAfOZRXW5DpQ+1J3nsQ== 2022-11-27T15:01:31.163 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2022-11-27T15:01:31.163 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQC6e4NjxXxjFxAA8FC/UdocH69QgR7lA3qxww== 2022-11-27T15:01:31.164 INFO:teuthology.orchestra.run.smithi039.stdout:[client.1] 2022-11-27T15:01:31.164 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQC6e4NjTja2GxAAzuIH1TMM2kCnADYJSsAIIw== 2022-11-27T15:01:31.165 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T15:01:31.261 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T15:01:31.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:01:31.363 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:01:31.415 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T15:01:31.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:01:31.517 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:01:31.567 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:01:31.616 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T15:01:31.666 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T15:01:31.720 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T15:01:31.773 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-11-27T15:01:31.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-11-27T15:01:31.803 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T15:01:31.904 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-11-27T15:01:31.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-11-27T15:01:32.000 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T15:01:32.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-11-27T15:01:32.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-11-27T15:01:32.197 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T15:01:32.306 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-11-27T15:01:32.334 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T15:01:32.392 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-11-27T15:01:32.393 INFO:tasks.ceph.mon.a:Restarting daemon 2022-11-27T15:01:32.393 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-11-27T15:01:32.436 INFO:tasks.ceph.mon.a:Started 2022-11-27T15:01:32.437 INFO:tasks.ceph.mon.b:Restarting daemon 2022-11-27T15:01:32.438 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-11-27T15:01:32.441 INFO:tasks.ceph.mon.b:Started 2022-11-27T15:01:32.442 INFO:tasks.ceph.mon.c:Restarting daemon 2022-11-27T15:01:32.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-11-27T15:01:32.445 INFO:tasks.ceph.mon.c:Started 2022-11-27T15:01:32.447 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-11-27T15:01:32.447 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-11-27T15:01:32.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-11-27T15:01:32.451 INFO:tasks.ceph.mgr.x:Started 2022-11-27T15:01:32.452 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-11-27T15:01:32.453 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-11-27T15:01:32.456 INFO:tasks.ceph.mgr.y:Started 2022-11-27T15:01:32.458 INFO:tasks.ceph:Setting crush tunables to default 2022-11-27T15:01:32.458 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd crush tunables default 2022-11-27T15:01:32.599 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.598+0000 7f344ba13700 1 Processor -- start 2022-11-27T15:01:32.600 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.598+0000 7f344ba13700 1 -- start start 2022-11-27T15:01:32.601 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f344ba13700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f344405be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:32.602 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f344ba13700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f344405d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:32.602 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f344ba13700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3444139620 con 0x7f344405ccd0 2022-11-27T15:01:32.602 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f344ba13700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344405d110 con 0x7f3444139f50 2022-11-27T15:01:32.602 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f344ba13700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f344413a400 con 0x7f3444139b70 2022-11-27T15:01:32.603 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f34497af700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f344405be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:32.603 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f3449fb0700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f344405d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:32.603 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f3448fae700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f344405ccd0 0x7f344405fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:35350/0 (socket says 172.21.15.39:35350) 2022-11-27T15:01:32.603 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f3448fae700 1 -- 172.21.15.39:0/3648433457 learned_addr learned my addr 172.21.15.39:0/3648433457 (peer_addr_for_me v1:172.21.15.39:0/0) 2022-11-27T15:01:32.604 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f3449fb0700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 msgr2=0x7f344405d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-11-27T15:01:32.604 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f34497af700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 msgr2=0x7f344405be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T15:01:32.604 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f3449fb0700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 msgr2=0x7f344405d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T15:01:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f34497af700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 msgr2=0x7f344405be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T15:01:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f3449fb0700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f344405d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-27T15:01:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f34497af700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f344405be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-27T15:01:32.605 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f34497af700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f344405be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-11-27T15:01:32.606 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.599+0000 7f3449fb0700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f344405d6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-11-27T15:01:32.802 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.800+0000 7f34497af700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f344405be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:32.803 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.800+0000 7f3449fb0700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f344405d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:32.803 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.800+0000 7f34497af700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 msgr2=0x7f344405be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-11-27T15:01:32.803 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.800+0000 7f34497af700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 msgr2=0x7f344405be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T15:01:32.804 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.800+0000 7f34497af700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f344405be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-27T15:01:32.804 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.800+0000 7f3449fb0700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 msgr2=0x7f344405d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T15:01:32.804 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.800+0000 7f3449fb0700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 msgr2=0x7f344405d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T15:01:32.805 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.800+0000 7f3449fb0700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f344405d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-27T15:01:32.805 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.800+0000 7f34497af700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f344405be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-11-27T15:01:32.805 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.800+0000 7f3449fb0700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f344405d6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-11-27T15:01:32.834 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762183943 0 0) 0x7f3444139620 con 0x7f344405ccd0 2022-11-27T15:01:32.835 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f342c003260 con 0x7f344405ccd0 2022-11-27T15:01:32.835 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 <== mon.0 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3500770596 0 0) 0x7f342c003260 con 0x7f344405ccd0 2022-11-27T15:01:32.835 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 msgr2=0x7f344405be80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T15:01:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f344405be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 msgr2=0x7f344405d6c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T15:01:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f344405d6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344405dbb0 con 0x7f344405ccd0 2022-11-27T15:01:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 <== mon.0 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2265032216 0 0) 0x7f3440001660 con 0x7f344405ccd0 2022-11-27T15:01:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 >> v1:172.21.15.39:6789/0 conn(0x7f344405ccd0 legacy=0x7f344405fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f342c003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342c003d30 0x7f342c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f342c0049f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344405d110 con 0x7f342c003d30 2022-11-27T15:01:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344413a400 con 0x7f3444139f50 2022-11-27T15:01:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f342c005030 con 0x7f3444139b70 2022-11-27T15:01:32.838 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f3449fb0700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f342c0049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:32.839 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.831+0000 7f34497af700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f342c003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:32.839 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.832+0000 7f3448fae700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342c003d30 0x7f342c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:32.839 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.832+0000 7f3449fb0700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 msgr2=0x7f342c003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:32.839 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.832+0000 7f3449fb0700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f342c003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.832+0000 7f3449fb0700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342c003d30 msgr2=0x7f342c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.832+0000 7f3449fb0700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342c003d30 0x7f342c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.832+0000 7f3449fb0700 1 -- 172.21.15.39:0/3648433457 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c003260 con 0x7f3444139b70 2022-11-27T15:01:32.840 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.832+0000 7f3449fb0700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f342c0049f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f343c00a000 tx=0x7f343c008040).ready entity=mon.2 client_cookie=feaf7dc46c19bfae server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:32.843 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.836+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f343c010630 con 0x7f3444139b70 2022-11-27T15:01:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.836+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c00c030 con 0x7f3444139b70 2022-11-27T15:01:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.836+0000 7f3433fff700 1 -- 172.21.15.39:0/3648433457 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f343c010980 con 0x7f3444139b70 2022-11-27T15:01:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.836+0000 7f344ba13700 1 -- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 msgr2=0x7f342c0049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.836+0000 7f344ba13700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f342c0049f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f343c00a000 tx=0x7f343c008040).stop 2022-11-27T15:01:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.836+0000 7f344ba13700 1 -- 172.21.15.39:0/3648433457 shutdown_connections 2022-11-27T15:01:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.836+0000 7f344ba13700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342c003d30 0x7f342c004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.836+0000 7f344ba13700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3444139b70 0x7f342c0049f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:32.845 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.836+0000 7f344ba13700 1 --2- 172.21.15.39:0/3648433457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139f50 0x7f342c003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:32.846 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.836+0000 7f344ba13700 1 -- 172.21.15.39:0/3648433457 >> 172.21.15.39:0/3648433457 conn(0x7f344406c640 msgr2=0x7f34440ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:32.846 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.837+0000 7f344ba13700 1 -- 172.21.15.39:0/3648433457 shutdown_connections 2022-11-27T15:01:32.846 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.837+0000 7f344ba13700 1 -- 172.21.15.39:0/3648433457 wait complete. 2022-11-27T15:01:32.846 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.838+0000 7f344ba13700 1 Processor -- start 2022-11-27T15:01:32.846 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.838+0000 7f344ba13700 1 -- start start 2022-11-27T15:01:32.847 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f344ba13700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f342c003d30 0x7f3444115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:32.847 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f344ba13700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344405ccd0 0x7f3444148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:32.847 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f344ba13700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139b70 0x7f34441491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:32.847 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f344ba13700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34440a5a10 con 0x7f344405ccd0 2022-11-27T15:01:32.848 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f344ba13700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34440a6290 con 0x7f3444139b70 2022-11-27T15:01:32.848 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f344ba13700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3444116640 con 0x7f342c003d30 2022-11-27T15:01:32.848 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f34497af700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f342c003d30 0x7f3444115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:32.848 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f34497af700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f342c003d30 0x7f3444115c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:33620/0 (socket says 172.21.15.39:33620) 2022-11-27T15:01:32.848 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f34497af700 1 -- 172.21.15.39:0/3734004470 learned_addr learned my addr 172.21.15.39:0/3734004470 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:32.849 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f3449fb0700 1 --2- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139b70 0x7f34441491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:32.849 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f34497af700 1 -- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139b70 msgr2=0x7f34441491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:32.849 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f34497af700 1 --2- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139b70 0x7f34441491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:32.849 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f34497af700 1 -- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344405ccd0 msgr2=0x7f3444148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f34497af700 1 --2- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344405ccd0 0x7f3444148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-11-27T15:01:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f34497af700 1 -- 172.21.15.39:0/3734004470 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3444116b20 con 0x7f342c003d30 2022-11-27T15:01:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.839+0000 7f34497af700 1 --2- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f342c003d30 0x7f3444115c70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f34440a5b50 tx=0x7f3434008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.840+0000 7f34327fc700 1 -- 172.21.15.39:0/3734004470 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3434014690 con 0x7f342c003d30 2022-11-27T15:01:32.850 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.840+0000 7f34327fc700 1 -- 172.21.15.39:0/3734004470 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34340147f0 con 0x7f342c003d30 2022-11-27T15:01:32.851 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.840+0000 7f344ba13700 1 -- 172.21.15.39:0/3734004470 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34440a1b50 con 0x7f342c003d30 2022-11-27T15:01:32.851 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.840+0000 7f34327fc700 1 -- 172.21.15.39:0/3734004470 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3434014b50 con 0x7f342c003d30 2022-11-27T15:01:32.851 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.840+0000 7f344ba13700 1 -- 172.21.15.39:0/3734004470 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34440a5550 con 0x7f342c003d30 2022-11-27T15:01:32.851 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.840+0000 7f34327fc700 1 -- 172.21.15.39:0/3734004470 <== mon.2 v2:172.21.15.39:3302/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f3434025070 con 0x7f342c003d30 2022-11-27T15:01:32.852 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.840+0000 7f34327fc700 1 -- 172.21.15.39:0/3734004470 <== mon.2 v2:172.21.15.39:3302/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f34340159c0 con 0x7f342c003d30 2022-11-27T15:01:32.852 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.841+0000 7f344ba13700 1 -- 172.21.15.39:0/3734004470 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344405c4f0 con 0x7f342c003d30 2022-11-27T15:01:32.853 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.843+0000 7f34327fc700 1 -- 172.21.15.39:0/3734004470 <== mon.2 v2:172.21.15.39:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f343403c870 con 0x7f342c003d30 2022-11-27T15:01:32.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.913+0000 7f344ba13700 1 -- 172.21.15.39:0/3734004470 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3444057a60 con 0x7f342c003d30 2022-11-27T15:01:32.932 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.930+0000 7f34327fc700 1 -- 172.21.15.39:0/3734004470 <== mon.2 v2:172.21.15.39:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f343402c070 con 0x7f342c003d30 2022-11-27T15:01:32.934 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.931+0000 7f344ba13700 1 -- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f342c003d30 msgr2=0x7f3444115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:32.935 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.931+0000 7f344ba13700 1 --2- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f342c003d30 0x7f3444115c70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f34440a5b50 tx=0x7f3434008040).stop 2022-11-27T15:01:32.935 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.932+0000 7f344ba13700 1 -- 172.21.15.39:0/3734004470 shutdown_connections 2022-11-27T15:01:32.935 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.932+0000 7f344ba13700 1 --2- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3444139b70 0x7f34441491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:32.935 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.932+0000 7f344ba13700 1 --2- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f342c003d30 0x7f3444115c70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:32.936 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.932+0000 7f344ba13700 1 --2- 172.21.15.39:0/3734004470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f344405ccd0 0x7f3444148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-11-27T15:01:32.936 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.932+0000 7f344ba13700 1 -- 172.21.15.39:0/3734004470 >> 172.21.15.39:0/3734004470 conn(0x7f344406c640 msgr2=0x7f34440a9790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:32.936 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.932+0000 7f344ba13700 1 -- 172.21.15.39:0/3734004470 shutdown_connections 2022-11-27T15:01:32.937 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:32.932+0000 7f344ba13700 1 -- 172.21.15.39:0/3734004470 wait complete. 2022-11-27T15:01:32.937 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2022-11-27T15:01:32.943 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-11-27T15:01:32.944 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:32.944 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-11-27T15:01:32.972 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:32.973 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-11-27T15:01:33.040 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:33.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-11-27T15:01:33.109 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2022-11-27T15:01:33.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-11-27T15:01:33.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new ba0fb473-e933-4ad1-a5f8-2cdd4138f3ff 0 2022-11-27T15:01:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.325+0000 7fbbf3cbe700 1 Processor -- start 2022-11-27T15:01:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.325+0000 7fbbf3cbe700 1 -- start start 2022-11-27T15:01:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf3cbe700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbbec099f40 0x7fbbec09b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf3cbe700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbec09ba10 0x7fbbec133110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf3cbe700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec133600 0x7fbbec05b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf3cbe700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbec068020 con 0x7fbbec099f40 2022-11-27T15:01:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf3cbe700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec0682c0 con 0x7fbbec09ba10 2022-11-27T15:01:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf3cbe700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec067360 con 0x7fbbec133600 2022-11-27T15:01:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf225b700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec133600 0x7fbbec05b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf225b700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec133600 0x7fbbec05b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:33648/0 (socket says 172.21.15.39:33648) 2022-11-27T15:01:33.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf225b700 1 -- 172.21.15.39:0/2909219242 learned_addr learned my addr 172.21.15.39:0/2909219242 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:33.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf1a5a700 1 --2- 172.21.15.39:0/2909219242 >> v2:172.21.15.39:3300/0 conn(0x7fbbec099f40 0x7fbbec09b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf1259700 1 --2- 172.21.15.39:0/2909219242 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbec09ba10 0x7fbbec133110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf225b700 1 -- 172.21.15.39:0/2909219242 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbec09ba10 msgr2=0x7fbbec133110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf225b700 1 --2- 172.21.15.39:0/2909219242 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbec09ba10 0x7fbbec133110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf225b700 1 -- 172.21.15.39:0/2909219242 >> v2:172.21.15.39:3300/0 conn(0x7fbbec099f40 msgr2=0x7fbbec09b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf225b700 1 --2- 172.21.15.39:0/2909219242 >> v2:172.21.15.39:3300/0 conn(0x7fbbec099f40 0x7fbbec09b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf225b700 1 -- 172.21.15.39:0/2909219242 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec05d6a0 con 0x7fbbec133600 2022-11-27T15:01:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.326+0000 7fbbf1a5a700 1 --2- 172.21.15.39:0/2909219242 >> v2:172.21.15.39:3300/0 conn(0x7fbbec099f40 0x7fbbec09b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf225b700 1 --2- 172.21.15.39:0/2909219242 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec133600 0x7fbbec05b970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fbbe40032f0 tx=0x7fbbe4009290).ready entity=mon.2 client_cookie=c2e212d504ffc8f6 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:33.338 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf0a58700 1 -- 172.21.15.39:0/2909219242 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbe400b8a0 con 0x7fbbec133600 2022-11-27T15:01:33.338 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf0a58700 1 -- 172.21.15.39:0/2909219242 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe400ba00 con 0x7fbbec133600 2022-11-27T15:01:33.339 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf0a58700 1 -- 172.21.15.39:0/2909219242 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbe400bb70 con 0x7fbbec133600 2022-11-27T15:01:33.339 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/2909219242 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec133600 msgr2=0x7fbbec05b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf3cbe700 1 --2- 172.21.15.39:0/2909219242 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec133600 0x7fbbec05b970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fbbe40032f0 tx=0x7fbbe4009290).stop 2022-11-27T15:01:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/2909219242 shutdown_connections 2022-11-27T15:01:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf3cbe700 1 --2- 172.21.15.39:0/2909219242 >> v2:172.21.15.39:3300/0 conn(0x7fbbec099f40 0x7fbbec09b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.341 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf3cbe700 1 --2- 172.21.15.39:0/2909219242 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec133600 0x7fbbec05b970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.341 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf3cbe700 1 --2- 172.21.15.39:0/2909219242 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbec09ba10 0x7fbbec133110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.341 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.327+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/2909219242 >> 172.21.15.39:0/2909219242 conn(0x7fbbec06c410 msgr2=0x7fbbec05ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:33.341 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.328+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/2909219242 shutdown_connections 2022-11-27T15:01:33.341 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.328+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/2909219242 wait complete. 2022-11-27T15:01:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.328+0000 7fbbf3cbe700 1 Processor -- start 2022-11-27T15:01:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.328+0000 7fbbf3cbe700 1 -- start start 2022-11-27T15:01:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf3cbe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec099f40 0x7fbbec092fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.342 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf3cbe700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec09ba10 0x7fbbec094100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.343 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf3cbe700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbec133600 0x7fbbec138840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.343 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf3cbe700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec139b20 con 0x7fbbec099f40 2022-11-27T15:01:33.343 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf3cbe700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec146680 con 0x7fbbec133600 2022-11-27T15:01:33.344 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf3cbe700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec145bf0 con 0x7fbbec09ba10 2022-11-27T15:01:33.344 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf1259700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec09ba10 0x7fbbec094100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.344 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf1259700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec09ba10 0x7fbbec094100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:33660/0 (socket says 172.21.15.39:33660) 2022-11-27T15:01:33.344 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf1259700 1 -- 172.21.15.39:0/3527365940 learned_addr learned my addr 172.21.15.39:0/3527365940 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf225b700 1 --2- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbec133600 0x7fbbec138840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf1a5a700 1 --2- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec099f40 0x7fbbec092fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.329+0000 7fbbf1259700 1 -- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbec133600 msgr2=0x7fbbec138840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbf1259700 1 --2- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbec133600 0x7fbbec138840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbf1259700 1 -- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec099f40 msgr2=0x7fbbec092fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbf1259700 1 --2- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec099f40 0x7fbbec092fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbf1259700 1 -- 172.21.15.39:0/3527365940 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe800b180 con 0x7fbbec09ba10 2022-11-27T15:01:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbf1259700 1 --2- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec09ba10 0x7fbbec094100 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fbbdc00a000 tx=0x7fbbdc008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbdaffd700 1 -- 172.21.15.39:0/3527365940 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbdc011780 con 0x7fbbec09ba10 2022-11-27T15:01:33.348 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbdaffd700 1 -- 172.21.15.39:0/3527365940 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc0118e0 con 0x7fbbec09ba10 2022-11-27T15:01:33.348 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/3527365940 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe4009530 con 0x7fbbec09ba10 2022-11-27T15:01:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbdaffd700 1 -- 172.21.15.39:0/3527365940 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbdc011c40 con 0x7fbbec09ba10 2022-11-27T15:01:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/3527365940 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbec093920 con 0x7fbbec09ba10 2022-11-27T15:01:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbdaffd700 1 -- 172.21.15.39:0/3527365940 <== mon.2 v2:172.21.15.39:3302/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fbbdc006020 con 0x7fbbec09ba10 2022-11-27T15:01:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.330+0000 7fbbdaffd700 1 -- 172.21.15.39:0/3527365940 <== mon.2 v2:172.21.15.39:3302/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbbdc005a40 con 0x7fbbec09ba10 2022-11-27T15:01:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.331+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/3527365940 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec099900 con 0x7fbbec09ba10 2022-11-27T15:01:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.333+0000 7fbbdaffd700 1 -- 172.21.15.39:0/3527365940 <== mon.2 v2:172.21.15.39:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fbbdc005c00 con 0x7fbbec09ba10 2022-11-27T15:01:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.410+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/3527365940 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "osd new", "uuid": "ba0fb473-e933-4ad1-a5f8-2cdd4138f3ff", "id": 0} v 0) v1 -- 0x7fbbec057a60 con 0x7fbbec09ba10 2022-11-27T15:01:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.428+0000 7fbbdaffd700 1 -- 172.21.15.39:0/3527365940 <== mon.2 v2:172.21.15.39:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ba0fb473-e933-4ad1-a5f8-2cdd4138f3ff", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbbdc02a0b0 con 0x7fbbec09ba10 2022-11-27T15:01:33.430 INFO:teuthology.orchestra.run.smithi039.stdout:0 2022-11-27T15:01:33.431 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.429+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec09ba10 msgr2=0x7fbbec094100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.431 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.429+0000 7fbbf3cbe700 1 --2- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec09ba10 0x7fbbec094100 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fbbdc00a000 tx=0x7fbbdc008040).stop 2022-11-27T15:01:33.431 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.429+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/3527365940 shutdown_connections 2022-11-27T15:01:33.432 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.429+0000 7fbbf3cbe700 1 --2- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbec099f40 0x7fbbec092fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.432 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.429+0000 7fbbf3cbe700 1 --2- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbec133600 0x7fbbec138840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.432 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.429+0000 7fbbf3cbe700 1 --2- 172.21.15.39:0/3527365940 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbbec09ba10 0x7fbbec094100 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.432 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.429+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/3527365940 >> 172.21.15.39:0/3527365940 conn(0x7fbbec06c410 msgr2=0x7fbbec0a8320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:33.432 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.429+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/3527365940 shutdown_connections 2022-11-27T15:01:33.433 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.429+0000 7fbbf3cbe700 1 -- 172.21.15.39:0/3527365940 wait complete. 2022-11-27T15:01:33.441 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new d3e5e074-117b-4420-8340-e241cb3e6553 1 2022-11-27T15:01:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.547+0000 7f3d1a7b9700 1 Processor -- start 2022-11-27T15:01:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.547+0000 7f3d1a7b9700 1 -- start start 2022-11-27T15:01:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.548+0000 7f3d1a7b9700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 0x7f3d140a1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.548+0000 7f3d1a7b9700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d140a2ea0 0x7f3d14139d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.548+0000 7f3d1a7b9700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d140a1080 con 0x7f3d1405b970 2022-11-27T15:01:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.548+0000 7f3d1a7b9700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140a1d50 con 0x7f3d140a2ea0 2022-11-27T15:01:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.548+0000 7f3d1a7b9700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140a0af0 con 0x7f3d14139920 2022-11-27T15:01:33.557 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.548+0000 7f3d13fff700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 0x7f3d140a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.557 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.548+0000 7f3d13fff700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 0x7f3d140a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:33662/0 (socket says 172.21.15.39:33662) 2022-11-27T15:01:33.557 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.548+0000 7f3d13fff700 1 -- 172.21.15.39:0/259943298 learned_addr learned my addr 172.21.15.39:0/259943298 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.548+0000 7f3d137fe700 1 --2- 172.21.15.39:0/259943298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d140a2ea0 0x7f3d14139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.558 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.549+0000 7f3d13fff700 1 -- 172.21.15.39:0/259943298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d140a2ea0 msgr2=0x7f3d14139d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:33.559 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.549+0000 7f3d13fff700 1 --2- 172.21.15.39:0/259943298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d140a2ea0 0x7f3d14139d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.559 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.549+0000 7f3d13fff700 1 -- 172.21.15.39:0/259943298 >> v1:172.21.15.39:6789/0 conn(0x7f3d1405b970 legacy=0x7f3d1405bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.560 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.549+0000 7f3d13fff700 1 -- 172.21.15.39:0/259943298 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d140a1fb0 con 0x7f3d14139920 2022-11-27T15:01:33.560 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.549+0000 7f3d137fe700 1 --2- 172.21.15.39:0/259943298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d140a2ea0 0x7f3d14139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:33.561 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.549+0000 7f3d13fff700 1 --2- 172.21.15.39:0/259943298 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 0x7f3d140a1860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f3d04002f70 tx=0x7f3d04006eb0).ready entity=mon.2 client_cookie=883cf802148b0dde server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:33.561 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.549+0000 7f3d12ffd700 1 -- 172.21.15.39:0/259943298 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d0400d570 con 0x7f3d14139920 2022-11-27T15:01:33.562 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.549+0000 7f3d12ffd700 1 -- 172.21.15.39:0/259943298 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d04000e90 con 0x7f3d14139920 2022-11-27T15:01:33.562 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.549+0000 7f3d12ffd700 1 -- 172.21.15.39:0/259943298 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d0400d900 con 0x7f3d14139920 2022-11-27T15:01:33.563 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.550+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/259943298 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 msgr2=0x7f3d140a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.563 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.550+0000 7f3d1a7b9700 1 --2- 172.21.15.39:0/259943298 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 0x7f3d140a1860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f3d04002f70 tx=0x7f3d04006eb0).stop 2022-11-27T15:01:33.564 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.550+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/259943298 shutdown_connections 2022-11-27T15:01:33.564 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.550+0000 7f3d1a7b9700 1 --2- 172.21.15.39:0/259943298 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 0x7f3d140a1860 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.565 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.550+0000 7f3d1a7b9700 1 --2- 172.21.15.39:0/259943298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d140a2ea0 0x7f3d14139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.565 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.550+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/259943298 >> 172.21.15.39:0/259943298 conn(0x7f3d140a8a70 msgr2=0x7f3d140a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:33.566 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.550+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/259943298 shutdown_connections 2022-11-27T15:01:33.566 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.550+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/259943298 wait complete. 2022-11-27T15:01:33.566 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.551+0000 7f3d1a7b9700 1 Processor -- start 2022-11-27T15:01:33.567 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.551+0000 7f3d1a7b9700 1 -- start start 2022-11-27T15:01:33.567 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d1a7b9700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d1405b970 0x7f3d14093190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.568 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d1a7b9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d140a2ea0 0x7f3d14145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.568 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d1a7b9700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 0x7f3d14135290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.568 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d1a7b9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14146560 con 0x7f3d140a2ea0 2022-11-27T15:01:33.569 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d1a7b9700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d14067130 con 0x7f3d1405b970 2022-11-27T15:01:33.569 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d1a7b9700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140672a0 con 0x7f3d14139920 2022-11-27T15:01:33.570 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d137fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d140a2ea0 0x7f3d14145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.570 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d137fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d140a2ea0 0x7f3d14145b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56742/0 (socket says 172.21.15.39:56742) 2022-11-27T15:01:33.570 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d137fe700 1 -- 172.21.15.39:0/1208313527 learned_addr learned my addr 172.21.15.39:0/1208313527 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:33.571 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d18d56700 1 --2- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 0x7f3d14135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.571 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d13fff700 1 --2- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d1405b970 0x7f3d14093190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.571 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d137fe700 1 -- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 msgr2=0x7f3d14135290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.572 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d137fe700 1 --2- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 0x7f3d14135290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.572 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d137fe700 1 -- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d1405b970 msgr2=0x7f3d14093190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.572 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d137fe700 1 --2- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d1405b970 0x7f3d14093190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.572 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d137fe700 1 -- 172.21.15.39:0/1208313527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0800b180 con 0x7f3d140a2ea0 2022-11-27T15:01:33.573 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d137fe700 1 --2- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d140a2ea0 0x7f3d14145b90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f3d14067590 tx=0x7f3d08006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:33.573 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d117fa700 1 -- 172.21.15.39:0/1208313527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d08010780 con 0x7f3d140a2ea0 2022-11-27T15:01:33.573 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d117fa700 1 -- 172.21.15.39:0/1208313527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d080108e0 con 0x7f3d140a2ea0 2022-11-27T15:01:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d117fa700 1 -- 172.21.15.39:0/1208313527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d08010c40 con 0x7f3d140a2ea0 2022-11-27T15:01:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.552+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/1208313527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0400b1e0 con 0x7f3d140a2ea0 2022-11-27T15:01:33.574 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.553+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/1208313527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d14066b80 con 0x7f3d140a2ea0 2022-11-27T15:01:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.553+0000 7f3d117fa700 1 -- 172.21.15.39:0/1208313527 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f3d0801c460 con 0x7f3d140a2ea0 2022-11-27T15:01:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.553+0000 7f3d117fa700 1 -- 172.21.15.39:0/1208313527 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f3d08010da0 con 0x7f3d140a2ea0 2022-11-27T15:01:33.575 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.554+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/1208313527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d14139450 con 0x7f3d140a2ea0 2022-11-27T15:01:33.577 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.558+0000 7f3d117fa700 1 -- 172.21.15.39:0/1208313527 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f3d08026430 con 0x7f3d140a2ea0 2022-11-27T15:01:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.634+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/1208313527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d3e5e074-117b-4420-8340-e241cb3e6553", "id": 1} v 0) v1 -- 0x7f3d141458b0 con 0x7f3d140a2ea0 2022-11-27T15:01:33.648 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.646+0000 7f3d117fa700 1 -- 172.21.15.39:0/1208313527 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d3e5e074-117b-4420-8340-e241cb3e6553", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3d141458b0 con 0x7f3d140a2ea0 2022-11-27T15:01:33.649 INFO:teuthology.orchestra.run.smithi039.stdout:1 2022-11-27T15:01:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.648+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d140a2ea0 msgr2=0x7f3d14145b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.648+0000 7f3d1a7b9700 1 --2- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d140a2ea0 0x7f3d14145b90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f3d14067590 tx=0x7f3d08006df0).stop 2022-11-27T15:01:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.648+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/1208313527 shutdown_connections 2022-11-27T15:01:33.654 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.648+0000 7f3d1a7b9700 1 --2- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f3d14139920 0x7f3d14135290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.654 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.648+0000 7f3d1a7b9700 1 --2- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3d1405b970 0x7f3d14093190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.654 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.648+0000 7f3d1a7b9700 1 --2- 172.21.15.39:0/1208313527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d140a2ea0 0x7f3d14145b90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.648+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/1208313527 >> 172.21.15.39:0/1208313527 conn(0x7f3d140a8a70 msgr2=0x7f3d140a5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.650+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/1208313527 shutdown_connections 2022-11-27T15:01:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.650+0000 7f3d1a7b9700 1 -- 172.21.15.39:0/1208313527 wait complete. 2022-11-27T15:01:33.664 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new 2d655d83-8b81-4419-a44f-f4833633c621 2 2022-11-27T15:01:33.772 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.770+0000 7f4f36355700 1 Processor -- start 2022-11-27T15:01:33.772 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.770+0000 7f4f36355700 1 -- start start 2022-11-27T15:01:33.778 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.771+0000 7f4f36355700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f300675f0 0x7f4f30062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.779 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.771+0000 7f4f36355700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4f3013a2c0 0x7f4f3013c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.779 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.771+0000 7f4f36355700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f300635b0 con 0x7f4f3005cdb0 2022-11-27T15:01:33.779 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.771+0000 7f4f36355700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f30061aa0 con 0x7f4f3013a2c0 2022-11-27T15:01:33.780 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.771+0000 7f4f36355700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f300622d0 con 0x7f4f300675f0 2022-11-27T15:01:33.780 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f348f2700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4f3013a2c0 0x7f4f3013c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.780 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2f7fe700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f4f3005cdb0 0x7f4f300a47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:35370/0 (socket says 172.21.15.39:35370) 2022-11-27T15:01:33.780 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2ffff700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f300675f0 0x7f4f30062fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.780 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2f7fe700 1 -- 172.21.15.39:0/2694963707 learned_addr learned my addr 172.21.15.39:0/2694963707 (peer_addr_for_me v1:172.21.15.39:0/0) 2022-11-27T15:01:33.781 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2ffff700 1 -- 172.21.15.39:0/2694963707 >> v2:172.21.15.39:3300/0 conn(0x7f4f3013a2c0 msgr2=0x7f4f3013c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:33.781 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2ffff700 1 --2- 172.21.15.39:0/2694963707 >> v2:172.21.15.39:3300/0 conn(0x7f4f3013a2c0 0x7f4f3013c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.781 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2ffff700 1 -- 172.21.15.39:0/2694963707 >> v1:172.21.15.39:6789/0 conn(0x7f4f3005cdb0 legacy=0x7f4f300a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.781 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2ffff700 1 -- 172.21.15.39:0/2694963707 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f300627d0 con 0x7f4f300675f0 2022-11-27T15:01:33.782 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2ffff700 1 --2- 172.21.15.39:0/2694963707 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f300675f0 0x7f4f30062fb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4f20002f70 tx=0x7f4f20006df0).ready entity=mon.2 client_cookie=ee6d14883d0d4e03 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:33.782 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2effd700 1 -- 172.21.15.39:0/2694963707 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f2000d570 con 0x7f4f300675f0 2022-11-27T15:01:33.782 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2effd700 1 -- 172.21.15.39:0/2694963707 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f20000e90 con 0x7f4f300675f0 2022-11-27T15:01:33.782 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.772+0000 7f4f2effd700 1 -- 172.21.15.39:0/2694963707 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f2000d8c0 con 0x7f4f300675f0 2022-11-27T15:01:33.783 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.773+0000 7f4f36355700 1 -- 172.21.15.39:0/2694963707 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f300675f0 msgr2=0x7f4f30062fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.784 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.773+0000 7f4f36355700 1 --2- 172.21.15.39:0/2694963707 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f300675f0 0x7f4f30062fb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4f20002f70 tx=0x7f4f20006df0).stop 2022-11-27T15:01:33.784 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.773+0000 7f4f36355700 1 -- 172.21.15.39:0/2694963707 shutdown_connections 2022-11-27T15:01:33.785 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.773+0000 7f4f36355700 1 --2- 172.21.15.39:0/2694963707 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f300675f0 0x7f4f30062fb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.785 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.773+0000 7f4f36355700 1 --2- 172.21.15.39:0/2694963707 >> v2:172.21.15.39:3300/0 conn(0x7f4f3013a2c0 0x7f4f3013c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.786 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.773+0000 7f4f36355700 1 -- 172.21.15.39:0/2694963707 >> 172.21.15.39:0/2694963707 conn(0x7f4f300a8a70 msgr2=0x7f4f300a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:33.786 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.773+0000 7f4f36355700 1 -- 172.21.15.39:0/2694963707 shutdown_connections 2022-11-27T15:01:33.786 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.773+0000 7f4f36355700 1 -- 172.21.15.39:0/2694963707 wait complete. 2022-11-27T15:01:33.786 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.774+0000 7f4f36355700 1 Processor -- start 2022-11-27T15:01:33.787 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.774+0000 7f4f36355700 1 -- start start 2022-11-27T15:01:33.787 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f36355700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300675f0 0x7f4f30137e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.787 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f36355700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f3013a2c0 0x7f4f30148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.788 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f36355700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f30138850 0x7f4f301494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.788 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f36355700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f30137760 con 0x7f4f300675f0 2022-11-27T15:01:33.788 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f36355700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f301356d0 con 0x7f4f30138850 2022-11-27T15:01:33.788 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f36355700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f30135970 con 0x7f4f3013a2c0 2022-11-27T15:01:33.789 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2f7fe700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f3013a2c0 0x7f4f30148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.789 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300675f0 0x7f4f30137e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.789 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2f7fe700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f3013a2c0 0x7f4f30148fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:33700/0 (socket says 172.21.15.39:33700) 2022-11-27T15:01:33.789 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2f7fe700 1 -- 172.21.15.39:0/3956564400 learned_addr learned my addr 172.21.15.39:0/3956564400 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:33.789 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f348f2700 1 --2- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f30138850 0x7f4f301494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.790 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2ffff700 1 -- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f3013a2c0 msgr2=0x7f4f30148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.790 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2ffff700 1 --2- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f3013a2c0 0x7f4f30148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.790 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2ffff700 1 -- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f30138850 msgr2=0x7f4f301494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.791 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2ffff700 1 --2- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f30138850 0x7f4f301494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.791 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2ffff700 1 -- 172.21.15.39:0/3956564400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2000b180 con 0x7f4f300675f0 2022-11-27T15:01:33.791 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2f7fe700 1 --2- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f3013a2c0 0x7f4f30148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:33.792 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.775+0000 7f4f2ffff700 1 --2- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300675f0 0x7f4f30137e80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f4f20002f70 tx=0x7f4f20014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:33.792 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.776+0000 7f4f2d7fa700 1 -- 172.21.15.39:0/3956564400 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f20013da0 con 0x7f4f300675f0 2022-11-27T15:01:33.792 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.776+0000 7f4f2d7fa700 1 -- 172.21.15.39:0/3956564400 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f20006780 con 0x7f4f300675f0 2022-11-27T15:01:33.792 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.776+0000 7f4f36355700 1 -- 172.21.15.39:0/3956564400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f30135c30 con 0x7f4f300675f0 2022-11-27T15:01:33.793 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.776+0000 7f4f2d7fa700 1 -- 172.21.15.39:0/3956564400 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f2001e5f0 con 0x7f4f300675f0 2022-11-27T15:01:33.793 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.776+0000 7f4f36355700 1 -- 172.21.15.39:0/3956564400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f30138370 con 0x7f4f300675f0 2022-11-27T15:01:33.793 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.777+0000 7f4f2d7fa700 1 -- 172.21.15.39:0/3956564400 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f4f20025030 con 0x7f4f300675f0 2022-11-27T15:01:33.794 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.777+0000 7f4f2d7fa700 1 -- 172.21.15.39:0/3956564400 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f4f200117f0 con 0x7f4f300675f0 2022-11-27T15:01:33.794 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.777+0000 7f4f36355700 1 -- 172.21.15.39:0/3956564400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f30057a60 con 0x7f4f300675f0 2022-11-27T15:01:33.795 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.779+0000 7f4f2d7fa700 1 -- 172.21.15.39:0/3956564400 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f4f20023070 con 0x7f4f300675f0 2022-11-27T15:01:33.859 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.857+0000 7f4f36355700 1 -- 172.21.15.39:0/3956564400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2d655d83-8b81-4419-a44f-f4833633c621", "id": 2} v 0) v1 -- 0x7f4f30066c90 con 0x7f4f300675f0 2022-11-27T15:01:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.860+0000 7f4f2d7fa700 1 -- 172.21.15.39:0/3956564400 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2d655d83-8b81-4419-a44f-f4833633c621", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4f20019030 con 0x7f4f300675f0 2022-11-27T15:01:33.862 INFO:teuthology.orchestra.run.smithi039.stdout:2 2022-11-27T15:01:33.864 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.861+0000 7f4f36355700 1 -- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300675f0 msgr2=0x7f4f30137e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.864 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.861+0000 7f4f36355700 1 --2- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300675f0 0x7f4f30137e80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f4f20002f70 tx=0x7f4f20014000).stop 2022-11-27T15:01:33.864 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.861+0000 7f4f36355700 1 -- 172.21.15.39:0/3956564400 shutdown_connections 2022-11-27T15:01:33.864 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.861+0000 7f4f36355700 1 --2- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300675f0 0x7f4f30137e80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.865 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.861+0000 7f4f36355700 1 --2- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f30138850 0x7f4f301494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.865 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.861+0000 7f4f36355700 1 --2- 172.21.15.39:0/3956564400 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f4f3013a2c0 0x7f4f30148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.865 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.861+0000 7f4f36355700 1 -- 172.21.15.39:0/3956564400 >> 172.21.15.39:0/3956564400 conn(0x7f4f300a8a70 msgr2=0x7f4f30063c60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:33.865 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.861+0000 7f4f36355700 1 -- 172.21.15.39:0/3956564400 shutdown_connections 2022-11-27T15:01:33.865 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.861+0000 7f4f36355700 1 -- 172.21.15.39:0/3956564400 wait complete. 2022-11-27T15:01:33.873 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new 1f4c9a3a-a40b-43ae-8bd1-7672f6d9a51f 3 2022-11-27T15:01:33.982 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.981+0000 7f6434424700 1 Processor -- start 2022-11-27T15:01:33.983 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.981+0000 7f6434424700 1 -- start start 2022-11-27T15:01:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f6434424700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f642c134460 0x7f642c0657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f6434424700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c134840 0x7f642c137b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f6434424700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642c133fe0 con 0x7f642c138020 2022-11-27T15:01:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f6434424700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c064c80 con 0x7f642c134460 2022-11-27T15:01:33.990 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f6434424700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f642c134240 con 0x7f642c134840 2022-11-27T15:01:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64329c1700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f642c138020 0x7f642c05b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:35378/0 (socket says 172.21.15.39:35378) 2022-11-27T15:01:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64329c1700 1 -- 172.21.15.39:0/1636394414 learned_addr learned my addr 172.21.15.39:0/1636394414 (peer_addr_for_me v1:172.21.15.39:0/0) 2022-11-27T15:01:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64319bf700 1 --2- 172.21.15.39:0/1636394414 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c134840 0x7f642c137b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64321c0700 1 --2- 172.21.15.39:0/1636394414 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f642c134460 0x7f642c0657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64311be700 1 -- 172.21.15.39:0/1636394414 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214807949 0 0) 0x7f642c133fe0 con 0x7f642c138020 2022-11-27T15:01:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64311be700 1 -- 172.21.15.39:0/1636394414 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6414003260 con 0x7f642c138020 2022-11-27T15:01:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64319bf700 1 -- 172.21.15.39:0/1636394414 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f642c134460 msgr2=0x7f642c0657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64319bf700 1 --2- 172.21.15.39:0/1636394414 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f642c134460 0x7f642c0657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.993 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64319bf700 1 -- 172.21.15.39:0/1636394414 >> v1:172.21.15.39:6789/0 conn(0x7f642c138020 legacy=0x7f642c05b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.993 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64319bf700 1 -- 172.21.15.39:0/1636394414 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c062c40 con 0x7f642c134840 2022-11-27T15:01:33.993 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.982+0000 7f64319bf700 1 --2- 172.21.15.39:0/1636394414 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c134840 0x7f642c137b30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f641c002f70 tx=0x7f641c006df0).ready entity=mon.2 client_cookie=5cdfb31d515344e4 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:33.993 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.983+0000 7f64311be700 1 -- 172.21.15.39:0/1636394414 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f641c00d570 con 0x7f642c134840 2022-11-27T15:01:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.983+0000 7f64311be700 1 -- 172.21.15.39:0/1636394414 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c000e90 con 0x7f642c134840 2022-11-27T15:01:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.983+0000 7f64311be700 1 -- 172.21.15.39:0/1636394414 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f641c00d8c0 con 0x7f642c134840 2022-11-27T15:01:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.983+0000 7f6434424700 1 -- 172.21.15.39:0/1636394414 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c134840 msgr2=0x7f642c137b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.983+0000 7f6434424700 1 --2- 172.21.15.39:0/1636394414 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c134840 0x7f642c137b30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f641c002f70 tx=0x7f641c006df0).stop 2022-11-27T15:01:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.983+0000 7f6434424700 1 -- 172.21.15.39:0/1636394414 shutdown_connections 2022-11-27T15:01:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.983+0000 7f6434424700 1 --2- 172.21.15.39:0/1636394414 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f642c134460 0x7f642c0657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.983+0000 7f6434424700 1 --2- 172.21.15.39:0/1636394414 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c134840 0x7f642c137b30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.983+0000 7f6434424700 1 -- 172.21.15.39:0/1636394414 >> 172.21.15.39:0/1636394414 conn(0x7f642c06c410 msgr2=0x7f642c05ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.984+0000 7f6434424700 1 -- 172.21.15.39:0/1636394414 shutdown_connections 2022-11-27T15:01:33.997 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.984+0000 7f6434424700 1 -- 172.21.15.39:0/1636394414 wait complete. 2022-11-27T15:01:33.997 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.985+0000 7f6434424700 1 Processor -- start 2022-11-27T15:01:33.997 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.985+0000 7f6434424700 1 -- start start 2022-11-27T15:01:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.985+0000 7f6434424700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f642c134460 0x7f642c066030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.985+0000 7f6434424700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f642c134840 0x7f642c0920b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.985+0000 7f6434424700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c138020 0x7f642c099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.985+0000 7f6434424700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c093550 con 0x7f642c134460 2022-11-27T15:01:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.985+0000 7f6434424700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c093fe0 con 0x7f642c134840 2022-11-27T15:01:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.985+0000 7f6434424700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f642c066650 con 0x7f642c138020 2022-11-27T15:01:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f64329c1700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c138020 0x7f642c099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f64329c1700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c138020 0x7f642c099810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:33708/0 (socket says 172.21.15.39:33708) 2022-11-27T15:01:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f64329c1700 1 -- 172.21.15.39:0/1825679110 learned_addr learned my addr 172.21.15.39:0/1825679110 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f64329c1700 1 -- 172.21.15.39:0/1825679110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f642c134840 msgr2=0x7f642c0920b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f64329c1700 1 --2- 172.21.15.39:0/1825679110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f642c134840 0x7f642c0920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-11-27T15:01:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f64329c1700 1 -- 172.21.15.39:0/1825679110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f642c134460 msgr2=0x7f642c066030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f64329c1700 1 --2- 172.21.15.39:0/1825679110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f642c134460 0x7f642c066030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-11-27T15:01:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f64329c1700 1 -- 172.21.15.39:0/1825679110 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c00b180 con 0x7f642c138020 2022-11-27T15:01:34.003 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f64329c1700 1 --2- 172.21.15.39:0/1825679110 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c138020 0x7f642c099810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f642c066b70 tx=0x7f6424007dc0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f641b7fe700 1 -- 172.21.15.39:0/1825679110 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6424005c70 con 0x7f642c138020 2022-11-27T15:01:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f641b7fe700 1 -- 172.21.15.39:0/1825679110 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6424005dd0 con 0x7f642c138020 2022-11-27T15:01:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f6434424700 1 -- 172.21.15.39:0/1825679110 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c0939a0 con 0x7f642c138020 2022-11-27T15:01:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f641b7fe700 1 -- 172.21.15.39:0/1825679110 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64240072c0 con 0x7f642c138020 2022-11-27T15:01:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.986+0000 7f6434424700 1 -- 172.21.15.39:0/1825679110 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c09b880 con 0x7f642c138020 2022-11-27T15:01:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.987+0000 7f641b7fe700 1 -- 172.21.15.39:0/1825679110 <== mon.2 v2:172.21.15.39:3302/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f642401a060 con 0x7f642c138020 2022-11-27T15:01:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.987+0000 7f641b7fe700 1 -- 172.21.15.39:0/1825679110 <== mon.2 v2:172.21.15.39:3302/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f642401c890 con 0x7f642c138020 2022-11-27T15:01:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.987+0000 7f6434424700 1 -- 172.21.15.39:0/1825679110 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642c093b90 con 0x7f642c138020 2022-11-27T15:01:34.007 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:33.992+0000 7f641b7fe700 1 -- 172.21.15.39:0/1825679110 <== mon.2 v2:172.21.15.39:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f6424018070 con 0x7f642c138020 2022-11-27T15:01:34.071 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.069+0000 7f6434424700 1 -- 172.21.15.39:0/1825679110 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "osd new", "uuid": "1f4c9a3a-a40b-43ae-8bd1-7672f6d9a51f", "id": 3} v 0) v1 -- 0x7f642c1374a0 con 0x7f642c138020 2022-11-27T15:01:34.088 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.086+0000 7f641b7fe700 1 -- 172.21.15.39:0/1825679110 <== mon.2 v2:172.21.15.39:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1f4c9a3a-a40b-43ae-8bd1-7672f6d9a51f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f642401cb70 con 0x7f642c138020 2022-11-27T15:01:34.089 INFO:teuthology.orchestra.run.smithi039.stdout:3 2022-11-27T15:01:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.087+0000 7f6434424700 1 -- 172.21.15.39:0/1825679110 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c138020 msgr2=0x7f642c099810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.087+0000 7f6434424700 1 --2- 172.21.15.39:0/1825679110 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c138020 0x7f642c099810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f642c066b70 tx=0x7f6424007dc0).stop 2022-11-27T15:01:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.087+0000 7f6434424700 1 -- 172.21.15.39:0/1825679110 shutdown_connections 2022-11-27T15:01:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.087+0000 7f6434424700 1 --2- 172.21.15.39:0/1825679110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f642c134460 0x7f642c066030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-11-27T15:01:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.087+0000 7f6434424700 1 --2- 172.21.15.39:0/1825679110 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f642c138020 0x7f642c099810 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.087+0000 7f6434424700 1 --2- 172.21.15.39:0/1825679110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f642c134840 0x7f642c0920b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-11-27T15:01:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.087+0000 7f6434424700 1 -- 172.21.15.39:0/1825679110 >> 172.21.15.39:0/1825679110 conn(0x7f642c06c410 msgr2=0x7f642c06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:34.092 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.088+0000 7f6434424700 1 -- 172.21.15.39:0/1825679110 shutdown_connections 2022-11-27T15:01:34.093 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.088+0000 7f6434424700 1 -- 172.21.15.39:0/1825679110 wait complete. 2022-11-27T15:01:34.100 INFO:tasks.ceph.osd.0:Restarting daemon 2022-11-27T15:01:34.101 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-11-27T15:01:34.103 INFO:tasks.ceph.osd.0:Started 2022-11-27T15:01:34.104 INFO:tasks.ceph.osd.1:Restarting daemon 2022-11-27T15:01:34.104 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-11-27T15:01:34.107 INFO:tasks.ceph.osd.1:Started 2022-11-27T15:01:34.107 INFO:tasks.ceph.osd.2:Restarting daemon 2022-11-27T15:01:34.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-11-27T15:01:34.110 INFO:tasks.ceph.osd.2:Started 2022-11-27T15:01:34.111 INFO:tasks.ceph.osd.3:Restarting daemon 2022-11-27T15:01:34.111 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-11-27T15:01:34.114 INFO:tasks.ceph.osd.3:Started 2022-11-27T15:01:34.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T15:01:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.249+0000 7f8fe0ea8700 1 Processor -- start 2022-11-27T15:01:34.255 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.250+0000 7f8fe0ea8700 1 -- start start 2022-11-27T15:01:34.255 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.250+0000 7f8fe0ea8700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0aaaf0 0x7f8fdc060f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.255 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.250+0000 7f8fe0ea8700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc05b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.255 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.250+0000 7f8fe0ea8700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8fdc0a6970 0x7f8fdc05be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.250+0000 7f8fe0ea8700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fdc1339e0 con 0x7f8fdc0a6970 2022-11-27T15:01:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.250+0000 7f8fe0ea8700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc139490 con 0x7f8fdc05fd70 2022-11-27T15:01:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.250+0000 7f8fe0ea8700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc139730 con 0x7f8fdc0aaaf0 2022-11-27T15:01:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fd9d9b700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc05b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fda59c700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0aaaf0 0x7f8fdc060f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fd9d9b700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc05b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:58772/0 (socket says 172.21.15.39:58772) 2022-11-27T15:01:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fda59c700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0aaaf0 0x7f8fdc060f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:33750/0 (socket says 172.21.15.39:33750) 2022-11-27T15:01:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fd9d9b700 1 -- 172.21.15.39:0/348382596 learned_addr learned my addr 172.21.15.39:0/348382596 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fdad9d700 1 --2- 172.21.15.39:0/348382596 >> v2:172.21.15.39:3300/0 conn(0x7f8fdc0a6970 0x7f8fdc05be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.258 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fda59c700 1 -- 172.21.15.39:0/348382596 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 msgr2=0x7f8fdc05b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:34.258 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fda59c700 1 --2- 172.21.15.39:0/348382596 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc05b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.258 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fda59c700 1 -- 172.21.15.39:0/348382596 >> v2:172.21.15.39:3300/0 conn(0x7f8fdc0a6970 msgr2=0x7f8fdc05be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:34.258 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fda59c700 1 --2- 172.21.15.39:0/348382596 >> v2:172.21.15.39:3300/0 conn(0x7f8fdc0a6970 0x7f8fdc05be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.258 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fda59c700 1 -- 172.21.15.39:0/348382596 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc1399d0 con 0x7f8fdc0aaaf0 2022-11-27T15:01:34.259 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fdad9d700 1 --2- 172.21.15.39:0/348382596 >> v2:172.21.15.39:3300/0 conn(0x7f8fdc0a6970 0x7f8fdc05be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-11-27T15:01:34.259 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fd9d9b700 1 --2- 172.21.15.39:0/348382596 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc05b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:34.259 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fda59c700 1 --2- 172.21.15.39:0/348382596 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0aaaf0 0x7f8fdc060f70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f8fdc060e50 tx=0x7f8fcc006df0).ready entity=mon.2 client_cookie=6abcfbc08f75fce0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:34.259 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fd959a700 1 -- 172.21.15.39:0/348382596 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fcc00d570 con 0x7f8fdc0aaaf0 2022-11-27T15:01:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fd959a700 1 -- 172.21.15.39:0/348382596 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc000e90 con 0x7f8fdc0aaaf0 2022-11-27T15:01:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.251+0000 7f8fd959a700 1 -- 172.21.15.39:0/348382596 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fcc00d8c0 con 0x7f8fdc0aaaf0 2022-11-27T15:01:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.252+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/348382596 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0aaaf0 msgr2=0x7f8fdc060f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.261 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.252+0000 7f8fe0ea8700 1 --2- 172.21.15.39:0/348382596 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0aaaf0 0x7f8fdc060f70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f8fdc060e50 tx=0x7f8fcc006df0).stop 2022-11-27T15:01:34.261 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.252+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/348382596 shutdown_connections 2022-11-27T15:01:34.262 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.252+0000 7f8fe0ea8700 1 --2- 172.21.15.39:0/348382596 >> v2:172.21.15.39:3300/0 conn(0x7f8fdc0a6970 0x7f8fdc05be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.262 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.252+0000 7f8fe0ea8700 1 --2- 172.21.15.39:0/348382596 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0aaaf0 0x7f8fdc060f70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.262 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.252+0000 7f8fe0ea8700 1 --2- 172.21.15.39:0/348382596 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc05b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.262 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.252+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/348382596 >> 172.21.15.39:0/348382596 conn(0x7f8fdc062820 msgr2=0x7f8fdc0aa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:34.262 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.252+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/348382596 shutdown_connections 2022-11-27T15:01:34.263 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.252+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/348382596 wait complete. 2022-11-27T15:01:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.253+0000 7f8fe0ea8700 1 Processor -- start 2022-11-27T15:01:34.267 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.253+0000 7f8fe0ea8700 1 -- start start 2022-11-27T15:01:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.254+0000 7f8fe0ea8700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.254+0000 7f8fe0ea8700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0a6970 0x7f8fdc149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.254+0000 7f8fe0ea8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fdc0aaaf0 0x7f8fdc149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.254+0000 7f8fe0ea8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc066be0 con 0x7f8fdc0aaaf0 2022-11-27T15:01:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.254+0000 7f8fe0ea8700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc146500 con 0x7f8fdc05fd70 2022-11-27T15:01:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.254+0000 7f8fe0ea8700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1467c0 con 0x7f8fdc0a6970 2022-11-27T15:01:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fda59c700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fdad9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fdc0aaaf0 0x7f8fdc149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.271 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fda59c700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc145cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:58786/0 (socket says 172.21.15.39:58786) 2022-11-27T15:01:34.272 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fda59c700 1 -- 172.21.15.39:0/4074365314 learned_addr learned my addr 172.21.15.39:0/4074365314 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:34.272 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fd9d9b700 1 --2- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0a6970 0x7f8fdc149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.273 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fda59c700 1 -- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0a6970 msgr2=0x7f8fdc149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.273 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fda59c700 1 --2- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0a6970 0x7f8fdc149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.274 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fda59c700 1 -- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fdc0aaaf0 msgr2=0x7f8fdc149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.274 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fda59c700 1 --2- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fdc0aaaf0 0x7f8fdc149730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.274 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fda59c700 1 -- 172.21.15.39:0/4074365314 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd000b1e0 con 0x7f8fdc05fd70 2022-11-27T15:01:34.275 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fdad9d700 1 --2- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fdc0aaaf0 0x7f8fdc149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:34.275 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fd9d9b700 1 --2- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0a6970 0x7f8fdc149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fda59c700 1 --2- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc145cf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f8fdc060e50 tx=0x7f8fcc014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fcb7fe700 1 -- 172.21.15.39:0/4074365314 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fcc00dea0 con 0x7f8fdc05fd70 2022-11-27T15:01:34.276 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fcb7fe700 1 -- 172.21.15.39:0/4074365314 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc013e90 con 0x7f8fdc05fd70 2022-11-27T15:01:34.277 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/4074365314 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc00b180 con 0x7f8fdc05fd70 2022-11-27T15:01:34.277 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fcb7fe700 1 -- 172.21.15.39:0/4074365314 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fcc01e5f0 con 0x7f8fdc05fd70 2022-11-27T15:01:34.278 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.255+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/4074365314 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc146260 con 0x7f8fdc05fd70 2022-11-27T15:01:34.278 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.256+0000 7f8fcb7fe700 1 -- 172.21.15.39:0/4074365314 <== mon.1 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f8fcc025030 con 0x7f8fdc05fd70 2022-11-27T15:01:34.278 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.256+0000 7f8fcb7fe700 1 -- 172.21.15.39:0/4074365314 <== mon.1 v2:172.21.15.39:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f8fcc01e920 con 0x7f8fdc05fd70 2022-11-27T15:01:34.279 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.257+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/4074365314 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fdc13a210 con 0x7f8fdc05fd70 2022-11-27T15:01:34.279 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.259+0000 7f8fcb7fe700 1 -- 172.21.15.39:0/4074365314 <== mon.1 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f8fcc023070 con 0x7f8fdc05fd70 2022-11-27T15:01:34.326 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.324+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/4074365314 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fdc057a60 con 0x7f8fdc05fd70 2022-11-27T15:01:34.326 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.325+0000 7f8fcb7fe700 1 -- 172.21.15.39:0/4074365314 <== mon.1 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+3913 (secure 0 0 0) 0x7f8fcc019030 con 0x7f8fdc05fd70 2022-11-27T15:01:34.327 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:34.328 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":6,"fsid":"fb73c991-5ff5-4983-821e-38ecbf33964f","created":"2022-11-27T15:01:32.796785+0000","modified":"2022-11-27T15:01:34.071070+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"ba0fb473-e933-4ad1-a5f8-2cdd4138f3ff","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d3e5e074-117b-4420-8340-e241cb3e6553","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2d655d83-8b81-4419-a44f-f4833633c621","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1f4c9a3a-a40b-43ae-8bd1-7672f6d9a51f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T15:01:34.330 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.326+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 msgr2=0x7f8fdc145cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.331 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.326+0000 7f8fe0ea8700 1 --2- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc145cf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f8fdc060e50 tx=0x7f8fcc014000).stop 2022-11-27T15:01:34.331 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.326+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/4074365314 shutdown_connections 2022-11-27T15:01:34.332 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.326+0000 7f8fe0ea8700 1 --2- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fdc0aaaf0 0x7f8fdc149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.332 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.326+0000 7f8fe0ea8700 1 --2- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fdc05fd70 0x7f8fdc145cf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.326+0000 7f8fe0ea8700 1 --2- 172.21.15.39:0/4074365314 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f8fdc0a6970 0x7f8fdc149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.326+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/4074365314 >> 172.21.15.39:0/4074365314 conn(0x7f8fdc062820 msgr2=0x7f8fdc137a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:34.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.326+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/4074365314 shutdown_connections 2022-11-27T15:01:34.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:34.326+0000 7f8fe0ea8700 1 -- 172.21.15.39:0/4074365314 wait complete. 2022-11-27T15:01:34.339 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-11-27T15:01:34.339 INFO:tasks.ceph:Waiting for OSDs to come up 2022-11-27T15:01:34.490 INFO:tasks.ceph.osd.0.smithi039.stderr:2022-11-27T15:01:34.489+0000 7fd3cde6ff00 -1 Falling back to public interface 2022-11-27T15:01:34.742 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T15:01:34.749 INFO:tasks.ceph.osd.3.smithi039.stderr:2022-11-27T15:01:34.747+0000 7f49258e0f00 -1 Falling back to public interface 2022-11-27T15:01:34.749 INFO:tasks.ceph.osd.1.smithi039.stderr:2022-11-27T15:01:34.748+0000 7ff5feb33f00 -1 Falling back to public interface 2022-11-27T15:01:34.760 INFO:tasks.ceph.osd.2.smithi039.stderr:2022-11-27T15:01:34.758+0000 7fc6a47a9f00 -1 Falling back to public interface 2022-11-27T15:01:34.860 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.859+0000 7facac538700 1 Processor -- start 2022-11-27T15:01:34.862 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.859+0000 7facac538700 1 -- start start 2022-11-27T15:01:34.863 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7facac538700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faca4137a20 0x7faca40a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.863 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7facac538700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4092610 0x7faca405c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.863 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7facac538700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7faca40a4320 0x7faca40a4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.863 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7facac538700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca4137e00 con 0x7faca4137a20 2022-11-27T15:01:34.864 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7facac538700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca4091e50 con 0x7faca4092610 2022-11-27T15:01:34.864 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7facac538700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faca413dc60 con 0x7faca40a4320 2022-11-27T15:01:34.864 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7faca9ad3700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4092610 0x7faca405c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.864 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7facaaad5700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7faca40a4320 0x7faca40a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.865 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7facaaad5700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7faca40a4320 0x7faca40a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:33804/0 (socket says 172.21.15.39:33804) 2022-11-27T15:01:34.865 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7faca9ad3700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4092610 0x7faca405c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:58852/0 (socket says 172.21.15.39:58852) 2022-11-27T15:01:34.865 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7facaaad5700 1 -- 172.21.15.39:0/3200179197 learned_addr learned my addr 172.21.15.39:0/3200179197 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:34.865 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7faca9ad3700 1 -- 172.21.15.39:0/3200179197 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7faca40a4320 msgr2=0x7faca40a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:34.866 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7faca9ad3700 1 --2- 172.21.15.39:0/3200179197 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7faca40a4320 0x7faca40a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.866 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7faca9ad3700 1 -- 172.21.15.39:0/3200179197 >> v2:172.21.15.39:3300/0 conn(0x7faca4137a20 msgr2=0x7faca40a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:34.866 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7faca9ad3700 1 --2- 172.21.15.39:0/3200179197 >> v2:172.21.15.39:3300/0 conn(0x7faca4137a20 0x7faca40a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-11-27T15:01:34.866 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.860+0000 7faca9ad3700 1 -- 172.21.15.39:0/3200179197 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca413df00 con 0x7faca4092610 2022-11-27T15:01:34.867 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.861+0000 7faca9ad3700 1 --2- 172.21.15.39:0/3200179197 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4092610 0x7faca405c0b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fac94002f70 tx=0x7fac94006df0).ready entity=mon.1 client_cookie=a6708c153439f4fc server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:34.867 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.861+0000 7faca92d2700 1 -- 172.21.15.39:0/3200179197 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9400d570 con 0x7faca4092610 2022-11-27T15:01:34.867 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.861+0000 7faca92d2700 1 -- 172.21.15.39:0/3200179197 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac94000e90 con 0x7faca4092610 2022-11-27T15:01:34.867 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.861+0000 7faca92d2700 1 -- 172.21.15.39:0/3200179197 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9400d900 con 0x7faca4092610 2022-11-27T15:01:34.875 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.862+0000 7facac538700 1 -- 172.21.15.39:0/3200179197 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4092610 msgr2=0x7faca405c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.876 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.862+0000 7facac538700 1 --2- 172.21.15.39:0/3200179197 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4092610 0x7faca405c0b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fac94002f70 tx=0x7fac94006df0).stop 2022-11-27T15:01:34.876 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.862+0000 7facac538700 1 -- 172.21.15.39:0/3200179197 shutdown_connections 2022-11-27T15:01:34.877 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.862+0000 7facac538700 1 --2- 172.21.15.39:0/3200179197 >> v2:172.21.15.39:3300/0 conn(0x7faca4137a20 0x7faca40a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-11-27T15:01:34.877 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.862+0000 7facac538700 1 --2- 172.21.15.39:0/3200179197 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7faca40a4320 0x7faca40a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.878 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.862+0000 7facac538700 1 --2- 172.21.15.39:0/3200179197 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4092610 0x7faca405c0b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.878 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.862+0000 7facac538700 1 -- 172.21.15.39:0/3200179197 >> 172.21.15.39:0/3200179197 conn(0x7faca40604e0 msgr2=0x7faca405b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:34.879 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.864+0000 7facac538700 1 -- 172.21.15.39:0/3200179197 shutdown_connections 2022-11-27T15:01:34.880 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.864+0000 7facac538700 1 -- 172.21.15.39:0/3200179197 wait complete. 2022-11-27T15:01:34.881 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.865+0000 7facac538700 1 Processor -- start 2022-11-27T15:01:34.881 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.866+0000 7facac538700 1 -- start start 2022-11-27T15:01:34.881 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.866+0000 7facac538700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7faca4092610 0x7faca4099440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.882 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.866+0000 7facac538700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faca40a4320 0x7faca4137100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.882 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.866+0000 7facac538700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4137a20 0x7faca4095110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:34.883 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.866+0000 7facac538700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca4096c60 con 0x7faca40a4320 2022-11-27T15:01:34.884 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.866+0000 7facac538700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca41409a0 con 0x7faca4137a20 2022-11-27T15:01:34.884 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.866+0000 7facac538700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faca4140c60 con 0x7faca4092610 2022-11-27T15:01:34.884 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7faca9ad3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faca40a4320 0x7faca4137100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.884 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7facaaad5700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4137a20 0x7faca4095110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:34.885 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7faca9ad3700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faca40a4320 0x7faca4137100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56880/0 (socket says 172.21.15.39:56880) 2022-11-27T15:01:34.885 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7faca9ad3700 1 -- 172.21.15.39:0/4218112603 learned_addr learned my addr 172.21.15.39:0/4218112603 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:34.885 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7faca9ad3700 1 -- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7faca4092610 msgr2=0x7faca4099440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.885 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7faca9ad3700 1 --2- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7faca4092610 0x7faca4099440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-11-27T15:01:34.886 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7faca9ad3700 1 -- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4137a20 msgr2=0x7faca4095110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.886 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7faca9ad3700 1 --2- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4137a20 0x7faca4095110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.886 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7faca9ad3700 1 -- 172.21.15.39:0/4218112603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9400b180 con 0x7faca40a4320 2022-11-27T15:01:34.886 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7facaaad5700 1 --2- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4137a20 0x7faca4095110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:34.887 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7faca9ad3700 1 --2- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faca40a4320 0x7faca4137100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fac94002f70 tx=0x7fac94014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:34.887 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.867+0000 7fac937fe700 1 -- 172.21.15.39:0/4218112603 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac94013ca0 con 0x7faca40a4320 2022-11-27T15:01:34.887 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.868+0000 7fac937fe700 1 -- 172.21.15.39:0/4218112603 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac94013e00 con 0x7faca40a4320 2022-11-27T15:01:34.887 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.868+0000 7facac538700 1 -- 172.21.15.39:0/4218112603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca413fdc0 con 0x7faca40a4320 2022-11-27T15:01:34.888 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.868+0000 7fac937fe700 1 -- 172.21.15.39:0/4218112603 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9401e5e0 con 0x7faca40a4320 2022-11-27T15:01:34.888 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.868+0000 7facac538700 1 -- 172.21.15.39:0/4218112603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca4096630 con 0x7faca40a4320 2022-11-27T15:01:34.888 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.868+0000 7fac937fe700 1 -- 172.21.15.39:0/4218112603 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 2) v1 ==== 20942+0+0 (secure 0 0 0) 0x7fac9401c070 con 0x7faca40a4320 2022-11-27T15:01:34.888 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.869+0000 7fac937fe700 1 -- 172.21.15.39:0/4218112603 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fac940062c0 con 0x7faca40a4320 2022-11-27T15:01:34.889 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.869+0000 7facac538700 1 -- 172.21.15.39:0/4218112603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca4140760 con 0x7faca40a4320 2022-11-27T15:01:34.889 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.873+0000 7fac937fe700 1 -- 172.21.15.39:0/4218112603 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7faca4140760 con 0x7faca40a4320 2022-11-27T15:01:34.939 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.938+0000 7facac538700 1 -- 172.21.15.39:0/4218112603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faca4140760 con 0x7faca40a4320 2022-11-27T15:01:34.941 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.939+0000 7fac937fe700 1 -- 172.21.15.39:0/4218112603 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+3913 (secure 0 0 0) 0x7faca4140760 con 0x7faca40a4320 2022-11-27T15:01:34.941 INFO:teuthology.misc.health.smithi039.stdout: 2022-11-27T15:01:34.942 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":6,"fsid":"fb73c991-5ff5-4983-821e-38ecbf33964f","created":"2022-11-27T15:01:32.796785+0000","modified":"2022-11-27T15:01:34.071070+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"ba0fb473-e933-4ad1-a5f8-2cdd4138f3ff","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d3e5e074-117b-4420-8340-e241cb3e6553","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2d655d83-8b81-4419-a44f-f4833633c621","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1f4c9a3a-a40b-43ae-8bd1-7672f6d9a51f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T15:01:34.943 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.940+0000 7facac538700 1 -- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faca40a4320 msgr2=0x7faca4137100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:34.944 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.940+0000 7facac538700 1 --2- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faca40a4320 0x7faca4137100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fac94002f70 tx=0x7fac94014000).stop 2022-11-27T15:01:34.944 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.940+0000 7facac538700 1 -- 172.21.15.39:0/4218112603 shutdown_connections 2022-11-27T15:01:34.944 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.940+0000 7facac538700 1 --2- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faca4137a20 0x7faca4095110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.944 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.940+0000 7facac538700 1 --2- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7faca4092610 0x7faca4099440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-11-27T15:01:34.945 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.940+0000 7facac538700 1 --2- 172.21.15.39:0/4218112603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faca40a4320 0x7faca4137100 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:34.945 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.940+0000 7facac538700 1 -- 172.21.15.39:0/4218112603 >> 172.21.15.39:0/4218112603 conn(0x7faca40604e0 msgr2=0x7faca40a4770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:34.945 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.940+0000 7facac538700 1 -- 172.21.15.39:0/4218112603 shutdown_connections 2022-11-27T15:01:34.945 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:34.940+0000 7facac538700 1 -- 172.21.15.39:0/4218112603 wait complete. 2022-11-27T15:01:34.951 DEBUG:teuthology.misc:0 of 4 OSDs are up 2022-11-27T15:01:35.570 INFO:tasks.ceph.osd.0.smithi039.stderr:2022-11-27T15:01:35.569+0000 7fd3cde6ff00 -1 osd.0 0 log_to_monitors {default=true} 2022-11-27T15:01:35.830 INFO:tasks.ceph.osd.1.smithi039.stderr:2022-11-27T15:01:35.828+0000 7ff5feb33f00 -1 osd.1 0 log_to_monitors {default=true} 2022-11-27T15:01:35.847 INFO:tasks.ceph.osd.3.smithi039.stderr:2022-11-27T15:01:35.846+0000 7f49258e0f00 -1 osd.3 0 log_to_monitors {default=true} 2022-11-27T15:01:35.854 INFO:tasks.ceph.osd.2.smithi039.stderr:2022-11-27T15:01:35.853+0000 7fc6a47a9f00 -1 osd.2 0 log_to_monitors {default=true} 2022-11-27T15:01:36.814 INFO:tasks.ceph.osd.0.smithi039.stderr:2022-11-27T15:01:36.812+0000 7fd3b6340700 -1 osd.0 0 waiting for initial osdmap 2022-11-27T15:01:36.823 INFO:tasks.ceph.osd.0.smithi039.stderr:2022-11-27T15:01:36.821+0000 7fd3c1d5d700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:01:37.815 INFO:tasks.ceph.osd.1.smithi039.stderr:2022-11-27T15:01:37.814+0000 7ff5e7004700 -1 osd.1 0 waiting for initial osdmap 2022-11-27T15:01:37.817 INFO:tasks.ceph.osd.3.smithi039.stderr:2022-11-27T15:01:37.816+0000 7f490ddb1700 -1 osd.3 0 waiting for initial osdmap 2022-11-27T15:01:37.819 INFO:tasks.ceph.osd.2.smithi039.stderr:2022-11-27T15:01:37.816+0000 7fc68cc7a700 -1 osd.2 0 waiting for initial osdmap 2022-11-27T15:01:37.822 INFO:tasks.ceph.osd.1.smithi039.stderr:2022-11-27T15:01:37.821+0000 7ff5f2a21700 -1 osd.1 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:01:37.826 INFO:tasks.ceph.osd.2.smithi039.stderr:2022-11-27T15:01:37.825+0000 7fc698697700 -1 osd.2 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:01:37.828 INFO:tasks.ceph.osd.3.smithi039.stderr:2022-11-27T15:01:37.827+0000 7f49197ce700 -1 osd.3 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T15:01:41.354 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T15:01:41.463 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.462+0000 7fb098d23700 1 Processor -- start 2022-11-27T15:01:41.464 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.462+0000 7fb098d23700 1 -- start start 2022-11-27T15:01:41.469 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb098d23700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb094097cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.470 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb098d23700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0941369d0 0x7fb094099f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.470 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb098d23700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09409ac90 con 0x7fb094136df0 2022-11-27T15:01:41.470 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb098d23700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09409aec0 con 0x7fb0941369d0 2022-11-27T15:01:41.470 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb098d23700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940991a0 con 0x7fb0941365f0 2022-11-27T15:01:41.471 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb091d9b700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0941369d0 0x7fb094099f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.471 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb09259c700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb094097cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.471 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb091d9b700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0941369d0 0x7fb094099f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:56302/0 (socket says 172.21.15.39:56302) 2022-11-27T15:01:41.472 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb091d9b700 1 -- 172.21.15.39:0/2870843704 learned_addr learned my addr 172.21.15.39:0/2870843704 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:41.472 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb09259c700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb094097cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:45488/0 (socket says 172.21.15.39:45488) 2022-11-27T15:01:41.472 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb091d9b700 1 -- 172.21.15.39:0/2870843704 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 msgr2=0x7fb094097cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:41.472 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb091d9b700 1 --2- 172.21.15.39:0/2870843704 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb094097cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.473 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb09159a700 1 -- 172.21.15.39:0/2870843704 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161038106 0 0) 0x7fb09409ac90 con 0x7fb094136df0 2022-11-27T15:01:41.473 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb091d9b700 1 -- 172.21.15.39:0/2870843704 >> v1:172.21.15.39:6789/0 conn(0x7fb094136df0 legacy=0x7fb0940961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:41.473 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb091d9b700 1 -- 172.21.15.39:0/2870843704 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb094068220 con 0x7fb0941369d0 2022-11-27T15:01:41.473 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb09259c700 1 --2- 172.21.15.39:0/2870843704 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb094097cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:41.474 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.463+0000 7fb091d9b700 1 --2- 172.21.15.39:0/2870843704 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0941369d0 0x7fb094099f40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fb088002f70 tx=0x7fb088006eb0).ready entity=mon.1 client_cookie=1b1e38be4dc64009 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:41.474 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb09159a700 1 -- 172.21.15.39:0/2870843704 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb08800d570 con 0x7fb0941369d0 2022-11-27T15:01:41.474 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb09159a700 1 -- 172.21.15.39:0/2870843704 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb088000e90 con 0x7fb0941369d0 2022-11-27T15:01:41.474 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb09159a700 1 -- 172.21.15.39:0/2870843704 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb08800d8c0 con 0x7fb0941369d0 2022-11-27T15:01:41.475 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb098d23700 1 -- 172.21.15.39:0/2870843704 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0941369d0 msgr2=0x7fb094099f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:41.475 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb098d23700 1 --2- 172.21.15.39:0/2870843704 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0941369d0 0x7fb094099f40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fb088002f70 tx=0x7fb088006eb0).stop 2022-11-27T15:01:41.475 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb098d23700 1 -- 172.21.15.39:0/2870843704 shutdown_connections 2022-11-27T15:01:41.475 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb098d23700 1 --2- 172.21.15.39:0/2870843704 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb094097cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.476 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb098d23700 1 --2- 172.21.15.39:0/2870843704 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0941369d0 0x7fb094099f40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.476 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb098d23700 1 -- 172.21.15.39:0/2870843704 >> 172.21.15.39:0/2870843704 conn(0x7fb09405b9a0 msgr2=0x7fb0940a8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:41.476 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb098d23700 1 -- 172.21.15.39:0/2870843704 shutdown_connections 2022-11-27T15:01:41.476 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.464+0000 7fb098d23700 1 -- 172.21.15.39:0/2870843704 wait complete. 2022-11-27T15:01:41.476 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.465+0000 7fb098d23700 1 Processor -- start 2022-11-27T15:01:41.477 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.465+0000 7fb098d23700 1 -- start start 2022-11-27T15:01:41.477 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb098d23700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb0941455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.477 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb098d23700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0941369d0 0x7fb0941489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.477 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb098d23700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb094136df0 0x7fb094148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.478 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb098d23700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb094066f60 con 0x7fb0941369d0 2022-11-27T15:01:41.478 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb098d23700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940670c0 con 0x7fb094136df0 2022-11-27T15:01:41.478 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb098d23700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb094066060 con 0x7fb0941365f0 2022-11-27T15:01:41.478 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb09259c700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb0941455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.479 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb09259c700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb0941455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:45496/0 (socket says 172.21.15.39:45496) 2022-11-27T15:01:41.479 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb091d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0941369d0 0x7fb0941489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.479 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb09259c700 1 -- 172.21.15.39:0/495404504 learned_addr learned my addr 172.21.15.39:0/495404504 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:41.479 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb092d9d700 1 --2- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb094136df0 0x7fb094148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.480 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb09259c700 1 -- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb094136df0 msgr2=0x7fb094148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:41.480 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb09259c700 1 --2- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb094136df0 0x7fb094148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.480 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb09259c700 1 -- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0941369d0 msgr2=0x7fb0941489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:41.480 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb09259c700 1 --2- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0941369d0 0x7fb0941489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.481 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb09259c700 1 -- 172.21.15.39:0/495404504 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08400b180 con 0x7fb0941365f0 2022-11-27T15:01:41.481 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb091d9b700 1 --2- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0941369d0 0x7fb0941489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:41.481 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb09259c700 1 --2- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb0941455c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb094065fb0 tx=0x7fb084006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:41.481 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.466+0000 7fb0837fe700 1 -- 172.21.15.39:0/495404504 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb08400c5f0 con 0x7fb0941365f0 2022-11-27T15:01:41.482 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.467+0000 7fb0837fe700 1 -- 172.21.15.39:0/495404504 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08400c750 con 0x7fb0941365f0 2022-11-27T15:01:41.482 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.467+0000 7fb098d23700 1 -- 172.21.15.39:0/495404504 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08800b1e0 con 0x7fb0941365f0 2022-11-27T15:01:41.482 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.467+0000 7fb0837fe700 1 -- 172.21.15.39:0/495404504 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb08400cab0 con 0x7fb0941365f0 2022-11-27T15:01:41.482 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.467+0000 7fb098d23700 1 -- 172.21.15.39:0/495404504 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb094066330 con 0x7fb0941365f0 2022-11-27T15:01:41.482 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.468+0000 7fb0837fe700 1 -- 172.21.15.39:0/495404504 <== mon.2 v2:172.21.15.39:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fb08400cc10 con 0x7fb0941365f0 2022-11-27T15:01:41.483 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.468+0000 7fb0837fe700 1 -- 172.21.15.39:0/495404504 <== mon.2 v2:172.21.15.39:3302/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2469+0+0 (secure 0 0 0) 0x7fb08404a980 con 0x7fb0941365f0 2022-11-27T15:01:41.483 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.468+0000 7fb098d23700 1 -- 172.21.15.39:0/495404504 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb094064510 con 0x7fb0941365f0 2022-11-27T15:01:41.484 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.471+0000 7fb0837fe700 1 -- 172.21.15.39:0/495404504 <== mon.2 v2:172.21.15.39:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fb0840336c0 con 0x7fb0941365f0 2022-11-27T15:01:41.589 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.588+0000 7fb098d23700 1 -- 172.21.15.39:0/495404504 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb094057a60 con 0x7fb0941365f0 2022-11-27T15:01:41.590 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.589+0000 7fb0837fe700 1 -- 172.21.15.39:0/495404504 <== mon.2 v2:172.21.15.39:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+6273 (secure 0 0 0) 0x7fb0840332b0 con 0x7fb0941365f0 2022-11-27T15:01:41.591 INFO:teuthology.misc.health.smithi039.stdout: 2022-11-27T15:01:41.592 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":12,"fsid":"fb73c991-5ff5-4983-821e-38ecbf33964f","created":"2022-11-27T15:01:32.796785+0000","modified":"2022-11-27T15:01:40.834224+0000","last_up_change":"2022-11-27T15:01:38.820289+0000","last_in_change":"2022-11-27T15:01:38.820289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-11-27T15:01:34.849418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"ba0fb473-e933-4ad1-a5f8-2cdd4138f3ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6800","nonce":105137}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6801","nonce":105137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6803","nonce":105137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6802","nonce":105137}]},"public_addr":"172.21.15.39:6800/105137","cluster_addr":"172.21.15.39:6801/105137","heartbeat_back_addr":"172.21.15.39:6803/105137","heartbeat_front_addr":"172.21.15.39:6802/105137","state":["exists","up"]},{"osd":1,"uuid":"d3e5e074-117b-4420-8340-e241cb3e6553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6805","nonce":105140}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6809","nonce":105140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6811","nonce":105140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6810","nonce":105140}]},"public_addr":"172.21.15.39:6805/105140","cluster_addr":"172.21.15.39:6809/105140","heartbeat_back_addr":"172.21.15.39:6811/105140","heartbeat_front_addr":"172.21.15.39:6810/105140","state":["exists","up"]},{"osd":2,"uuid":"2d655d83-8b81-4419-a44f-f4833633c621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6812","nonce":105139}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6813","nonce":105139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6815","nonce":105139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6814","nonce":105139}]},"public_addr":"172.21.15.39:6812/105139","cluster_addr":"172.21.15.39:6813/105139","heartbeat_back_addr":"172.21.15.39:6815/105139","heartbeat_front_addr":"172.21.15.39:6814/105139","state":["exists","up"]},{"osd":3,"uuid":"1f4c9a3a-a40b-43ae-8bd1-7672f6d9a51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6804","nonce":105138}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6806","nonce":105138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6808","nonce":105138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6807","nonce":105138}]},"public_addr":"172.21.15.39:6804/105138","cluster_addr":"172.21.15.39:6806/105138","heartbeat_back_addr":"172.21.15.39:6808/105138","heartbeat_front_addr":"172.21.15.39:6807/105138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.532356+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.825983+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.875364+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.883149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T15:01:41.593 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.590+0000 7fb098d23700 1 -- 172.21.15.39:0/495404504 >> v1:172.21.15.39:6816/104537 conn(0x7fb070038da0 legacy=0x7fb07003b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:41.593 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.590+0000 7fb098d23700 1 -- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 msgr2=0x7fb0941455c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:41.593 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.590+0000 7fb098d23700 1 --2- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb0941455c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb094065fb0 tx=0x7fb084006df0).stop 2022-11-27T15:01:41.594 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.590+0000 7fb098d23700 1 -- 172.21.15.39:0/495404504 shutdown_connections 2022-11-27T15:01:41.594 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.590+0000 7fb098d23700 1 --2- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb094136df0 0x7fb094148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.594 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.590+0000 7fb098d23700 1 --2- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fb0941365f0 0x7fb0941455c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.594 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.590+0000 7fb098d23700 1 --2- 172.21.15.39:0/495404504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0941369d0 0x7fb0941489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.595 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.590+0000 7fb098d23700 1 -- 172.21.15.39:0/495404504 >> 172.21.15.39:0/495404504 conn(0x7fb09405b9a0 msgr2=0x7fb0941357f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:41.595 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.590+0000 7fb098d23700 1 -- 172.21.15.39:0/495404504 shutdown_connections 2022-11-27T15:01:41.595 INFO:teuthology.misc.health.smithi039.stderr:2022-11-27T15:01:41.590+0000 7fb098d23700 1 -- 172.21.15.39:0/495404504 wait complete. 2022-11-27T15:01:41.601 DEBUG:teuthology.misc:4 of 4 OSDs are up 2022-11-27T15:01:41.601 INFO:tasks.ceph:Creating RBD pool 2022-11-27T15:01:41.602 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd pool create rbd 8 2022-11-27T15:01:41.718 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.707+0000 7ff41e883700 1 Processor -- start 2022-11-27T15:01:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.707+0000 7ff41e883700 1 -- start start 2022-11-27T15:01:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.707+0000 7ff41e883700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff418099f40 0x7ff41809b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.707+0000 7ff41e883700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff41809ba10 0x7ff418133110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.707+0000 7ff41e883700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 0x7ff41805b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.707+0000 7ff41e883700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff418068020 con 0x7ff41809ba10 2022-11-27T15:01:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.707+0000 7ff41e883700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180682c0 con 0x7ff418099f40 2022-11-27T15:01:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.707+0000 7ff41e883700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff418067360 con 0x7ff418133600 2022-11-27T15:01:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff4177fe700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff41809ba10 0x7ff418133110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff41ce20700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 0x7ff41805b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.723 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff4177fe700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff41809ba10 0x7ff418133110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38242/0 (socket says 172.21.15.39:38242) 2022-11-27T15:01:41.723 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff41ce20700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 0x7ff41805b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:45500/0 (socket says 172.21.15.39:45500) 2022-11-27T15:01:41.723 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff4177fe700 1 -- 172.21.15.39:0/4016550403 learned_addr learned my addr 172.21.15.39:0/4016550403 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff417fff700 1 --2- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff418099f40 0x7ff41809b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff4177fe700 1 -- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 msgr2=0x7ff41805b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff4177fe700 1 --2- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 0x7ff41805b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff4177fe700 1 -- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff418099f40 msgr2=0x7ff41809b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:41.726 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff4177fe700 1 --2- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff418099f40 0x7ff41809b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.726 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff4177fe700 1 -- 172.21.15.39:0/4016550403 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41805d6a0 con 0x7ff41809ba10 2022-11-27T15:01:41.726 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff417fff700 1 --2- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff418099f40 0x7ff41809b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff4177fe700 1 --2- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff41809ba10 0x7ff418133110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff40c002f70 tx=0x7ff40c006eb0).ready entity=mon.0 client_cookie=d18b66f3e7fe05f6 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff416ffd700 1 -- 172.21.15.39:0/4016550403 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff40c00d570 con 0x7ff41809ba10 2022-11-27T15:01:41.728 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff416ffd700 1 -- 172.21.15.39:0/4016550403 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40c000e90 con 0x7ff41809ba10 2022-11-27T15:01:41.728 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.708+0000 7ff416ffd700 1 -- 172.21.15.39:0/4016550403 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff40c00d900 con 0x7ff41809ba10 2022-11-27T15:01:41.729 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.709+0000 7ff41e883700 1 -- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff41809ba10 msgr2=0x7ff418133110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:41.729 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.709+0000 7ff41e883700 1 --2- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff41809ba10 0x7ff418133110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff40c002f70 tx=0x7ff40c006eb0).stop 2022-11-27T15:01:41.729 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.709+0000 7ff41e883700 1 -- 172.21.15.39:0/4016550403 shutdown_connections 2022-11-27T15:01:41.730 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.709+0000 7ff41e883700 1 --2- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 0x7ff41805b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.730 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.709+0000 7ff41e883700 1 --2- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff418099f40 0x7ff41809b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.731 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.709+0000 7ff41e883700 1 --2- 172.21.15.39:0/4016550403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff41809ba10 0x7ff418133110 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.731 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.709+0000 7ff41e883700 1 -- 172.21.15.39:0/4016550403 >> 172.21.15.39:0/4016550403 conn(0x7ff41806c410 msgr2=0x7ff41805ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:41.732 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.709+0000 7ff41e883700 1 -- 172.21.15.39:0/4016550403 shutdown_connections 2022-11-27T15:01:41.732 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.709+0000 7ff41e883700 1 -- 172.21.15.39:0/4016550403 wait complete. 2022-11-27T15:01:41.733 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.710+0000 7ff41e883700 1 Processor -- start 2022-11-27T15:01:41.733 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.710+0000 7ff41e883700 1 -- start start 2022-11-27T15:01:41.733 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.710+0000 7ff41e883700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff418099f40 0x7ff418092c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.734 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.710+0000 7ff41e883700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff41809ba10 0x7ff418136240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.734 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.710+0000 7ff41e883700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 0x7ff4181458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:41.735 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.710+0000 7ff41e883700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180932b0 con 0x7ff418099f40 2022-11-27T15:01:41.735 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.710+0000 7ff41e883700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff418145470 con 0x7ff41809ba10 2022-11-27T15:01:41.736 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.710+0000 7ff41e883700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4181368a0 con 0x7ff418133600 2022-11-27T15:01:41.736 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff417fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff418099f40 0x7ff418092c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.736 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff4177fe700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff41809ba10 0x7ff418136240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.737 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff417fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff418099f40 0x7ff418092c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38250/0 (socket says 172.21.15.39:38250) 2022-11-27T15:01:41.737 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff417fff700 1 -- 172.21.15.39:0/2800622962 learned_addr learned my addr 172.21.15.39:0/2800622962 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:41.738 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff41ce20700 1 --2- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 0x7ff4181458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:41.738 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff417fff700 1 -- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 msgr2=0x7ff4181458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:41.739 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff417fff700 1 --2- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 0x7ff4181458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.739 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff417fff700 1 -- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff41809ba10 msgr2=0x7ff418136240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:41.739 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff417fff700 1 --2- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff41809ba10 0x7ff418136240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:41.740 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff417fff700 1 -- 172.21.15.39:0/2800622962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40800b180 con 0x7ff418099f40 2022-11-27T15:01:41.740 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff4177fe700 1 --2- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff41809ba10 0x7ff418136240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:41.741 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff417fff700 1 --2- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff418099f40 0x7ff418092c70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff40800c000 tx=0x7ff408006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:41.741 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff4157fa700 1 -- 172.21.15.39:0/2800622962 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff40800c7e0 con 0x7ff418099f40 2022-11-27T15:01:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff4157fa700 1 -- 172.21.15.39:0/2800622962 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40800c940 con 0x7ff418099f40 2022-11-27T15:01:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff4157fa700 1 -- 172.21.15.39:0/2800622962 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff40800cca0 con 0x7ff418099f40 2022-11-27T15:01:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff41e883700 1 -- 172.21.15.39:0/2800622962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c00b1e0 con 0x7ff418099f40 2022-11-27T15:01:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.711+0000 7ff41e883700 1 -- 172.21.15.39:0/2800622962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff418091db0 con 0x7ff418099f40 2022-11-27T15:01:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.712+0000 7ff4157fa700 1 -- 172.21.15.39:0/2800622962 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7ff40801c460 con 0x7ff418099f40 2022-11-27T15:01:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.712+0000 7ff41e883700 1 -- 172.21.15.39:0/2800622962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff418136ac0 con 0x7ff418099f40 2022-11-27T15:01:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.712+0000 7ff4157fa700 1 -- 172.21.15.39:0/2800622962 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2469+0+0 (secure 0 0 0) 0x7ff40804a8f0 con 0x7ff418099f40 2022-11-27T15:01:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.716+0000 7ff4157fa700 1 -- 172.21.15.39:0/2800622962 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7ff4080336d0 con 0x7ff418099f40 2022-11-27T15:01:41.835 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:41.832+0000 7ff41e883700 1 -- 172.21.15.39:0/2800622962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7ff418099900 con 0x7ff418099f40 2022-11-27T15:01:42.097 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.096+0000 7ff4157fa700 1 -- 172.21.15.39:0/2800622962 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (secure 0 0 0) 0x7ff40800ce00 con 0x7ff418099f40 2022-11-27T15:01:42.099 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.098+0000 7ff41e883700 1 -- 172.21.15.39:0/2800622962 >> v1:172.21.15.39:6816/104537 conn(0x7ff3f8038dd0 legacy=0x7ff3f803b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:42.100 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.098+0000 7ff41e883700 1 -- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff418099f40 msgr2=0x7ff418092c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:42.100 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.098+0000 7ff41e883700 1 --2- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff418099f40 0x7ff418092c70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff40800c000 tx=0x7ff408006df0).stop 2022-11-27T15:01:42.100 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.098+0000 7ff41e883700 1 -- 172.21.15.39:0/2800622962 shutdown_connections 2022-11-27T15:01:42.101 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.098+0000 7ff41e883700 1 --2- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff418099f40 0x7ff418092c70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:42.101 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.098+0000 7ff41e883700 1 --2- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7ff418133600 0x7ff4181458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:42.101 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.098+0000 7ff41e883700 1 --2- 172.21.15.39:0/2800622962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff41809ba10 0x7ff418136240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.098+0000 7ff41e883700 1 -- 172.21.15.39:0/2800622962 >> 172.21.15.39:0/2800622962 conn(0x7ff41806c410 msgr2=0x7ff418132ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.098+0000 7ff41e883700 1 -- 172.21.15.39:0/2800622962 shutdown_connections 2022-11-27T15:01:42.102 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.098+0000 7ff41e883700 1 -- 172.21.15.39:0/2800622962 wait complete. 2022-11-27T15:01:42.103 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'rbd' created 2022-11-27T15:01:42.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-11-27T15:01:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.220+0000 7faceec3f700 1 Processor -- start 2022-11-27T15:01:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7faceec3f700 1 -- start start 2022-11-27T15:01:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7faceec3f700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7face8133350 0x7face8133730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:42.226 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7faceec3f700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face805cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:42.226 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7faceec3f700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face8139580 con 0x7face805fd70 2022-11-27T15:01:42.226 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7faceec3f700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face805c250 con 0x7face8133350 2022-11-27T15:01:42.227 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7faceec3f700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face81397b0 con 0x7face805d5d0 2022-11-27T15:01:42.227 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7face7fff700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face805cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:42.227 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7facec9db700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7face8133350 0x7face8133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:42.227 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7face7fff700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face805cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:56346/0 (socket says 172.21.15.39:56346) 2022-11-27T15:01:42.228 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7facec9db700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7face8133350 0x7face8133730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38266/0 (socket says 172.21.15.39:38266) 2022-11-27T15:01:42.228 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7faced1dc700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7face805fd70 0x7face8060200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56774/0 (socket says 172.21.15.39:56774) 2022-11-27T15:01:42.228 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.221+0000 7face7fff700 1 -- 172.21.15.39:0/88796954 learned_addr learned my addr 172.21.15.39:0/88796954 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:42.228 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7face7fff700 1 -- 172.21.15.39:0/88796954 >> v2:172.21.15.39:3300/0 conn(0x7face8133350 msgr2=0x7face8133730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:42.229 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7face7fff700 1 --2- 172.21.15.39:0/88796954 >> v2:172.21.15.39:3300/0 conn(0x7face8133350 0x7face8133730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:42.229 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7face7fff700 1 -- 172.21.15.39:0/88796954 >> v1:172.21.15.39:6789/0 conn(0x7face805fd70 legacy=0x7face8060200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:42.229 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7face7fff700 1 -- 172.21.15.39:0/88796954 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face8139a50 con 0x7face805d5d0 2022-11-27T15:01:42.229 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7face77fe700 1 -- 172.21.15.39:0/88796954 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489727666 0 0) 0x7face8139580 con 0x7face805fd70 2022-11-27T15:01:42.229 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7facec9db700 1 --2- 172.21.15.39:0/88796954 >> v2:172.21.15.39:3300/0 conn(0x7face8133350 0x7face8133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:42.230 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7face7fff700 1 --2- 172.21.15.39:0/88796954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face805cb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7facd4002f70 tx=0x7facd4006eb0).ready entity=mon.1 client_cookie=a06e2d7dc1231087 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:42.230 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7face77fe700 1 -- 172.21.15.39:0/88796954 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facd400d570 con 0x7face805d5d0 2022-11-27T15:01:42.230 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7face77fe700 1 -- 172.21.15.39:0/88796954 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd4000e90 con 0x7face805d5d0 2022-11-27T15:01:42.230 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7face77fe700 1 -- 172.21.15.39:0/88796954 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facd400d8c0 con 0x7face805d5d0 2022-11-27T15:01:42.230 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7faceec3f700 1 -- 172.21.15.39:0/88796954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 msgr2=0x7face805cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:42.231 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.222+0000 7faceec3f700 1 --2- 172.21.15.39:0/88796954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face805cb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7facd4002f70 tx=0x7facd4006eb0).stop 2022-11-27T15:01:42.231 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.223+0000 7faceec3f700 1 -- 172.21.15.39:0/88796954 shutdown_connections 2022-11-27T15:01:42.231 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.223+0000 7faceec3f700 1 --2- 172.21.15.39:0/88796954 >> v2:172.21.15.39:3300/0 conn(0x7face8133350 0x7face8133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:42.231 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.223+0000 7faceec3f700 1 --2- 172.21.15.39:0/88796954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face805cb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:42.232 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.223+0000 7faceec3f700 1 -- 172.21.15.39:0/88796954 >> 172.21.15.39:0/88796954 conn(0x7face8066e50 msgr2=0x7face80a8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:42.232 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.223+0000 7faceec3f700 1 -- 172.21.15.39:0/88796954 shutdown_connections 2022-11-27T15:01:42.232 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.223+0000 7faceec3f700 1 -- 172.21.15.39:0/88796954 wait complete. 2022-11-27T15:01:42.232 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.223+0000 7faceec3f700 1 Processor -- start 2022-11-27T15:01:42.232 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.223+0000 7faceec3f700 1 -- start start 2022-11-27T15:01:42.235 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7faceec3f700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face80a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:42.236 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7faceec3f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face805fd70 0x7face80a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:42.236 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7faceec3f700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7face8133350 0x7face8145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:42.236 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7faceec3f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face8130b10 con 0x7face805fd70 2022-11-27T15:01:42.236 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7faceec3f700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face80a3770 con 0x7face805d5d0 2022-11-27T15:01:42.237 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7faceec3f700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7face80a17b0 con 0x7face8133350 2022-11-27T15:01:42.237 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7face7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face805fd70 0x7face80a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:42.237 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7facec9db700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face80a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:42.237 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7faced1dc700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7face8133350 0x7face8145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:42.238 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7face7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face805fd70 0x7face80a2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38278/0 (socket says 172.21.15.39:38278) 2022-11-27T15:01:42.238 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.224+0000 7face7fff700 1 -- 172.21.15.39:0/2655706375 learned_addr learned my addr 172.21.15.39:0/2655706375 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:42.238 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7face7fff700 1 -- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7face8133350 msgr2=0x7face8145a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:42.238 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7face7fff700 1 --2- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7face8133350 0x7face8145a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:42.239 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7face7fff700 1 -- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 msgr2=0x7face80a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:42.239 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7face7fff700 1 --2- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face80a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:42.239 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7face7fff700 1 -- 172.21.15.39:0/2655706375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc00b180 con 0x7face805fd70 2022-11-27T15:01:42.239 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7facec9db700 1 --2- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face80a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:42.240 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7face7fff700 1 --2- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face805fd70 0x7face80a2650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7facd4002f70 tx=0x7facd4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:42.240 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7face5ffb700 1 -- 172.21.15.39:0/2655706375 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facd400d8f0 con 0x7face805fd70 2022-11-27T15:01:42.240 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7face5ffb700 1 -- 172.21.15.39:0/2655706375 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd4011970 con 0x7face805fd70 2022-11-27T15:01:42.240 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7faceec3f700 1 -- 172.21.15.39:0/2655706375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd400b1e0 con 0x7face805fd70 2022-11-27T15:01:42.240 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7face5ffb700 1 -- 172.21.15.39:0/2655706375 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facd4011cd0 con 0x7face805fd70 2022-11-27T15:01:42.241 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.225+0000 7faceec3f700 1 -- 172.21.15.39:0/2655706375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face8149e30 con 0x7face805fd70 2022-11-27T15:01:42.241 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.226+0000 7face5ffb700 1 -- 172.21.15.39:0/2655706375 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7facd4025030 con 0x7face805fd70 2022-11-27T15:01:42.241 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.226+0000 7faceec3f700 1 -- 172.21.15.39:0/2655706375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face805d060 con 0x7face805fd70 2022-11-27T15:01:42.241 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.226+0000 7face5ffb700 1 -- 172.21.15.39:0/2655706375 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2811+0+0 (secure 0 0 0) 0x7facd404b5d0 con 0x7face805fd70 2022-11-27T15:01:42.241 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.229+0000 7face5ffb700 1 -- 172.21.15.39:0/2655706375 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7facd4028820 con 0x7face805fd70 2022-11-27T15:01:42.348 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:42.347+0000 7faceec3f700 1 -- 172.21.15.39:0/2655706375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7face8057a60 con 0x7face805fd70 2022-11-27T15:01:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.109+0000 7face5ffb700 1 -- 172.21.15.39:0/2655706375 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 173+0+0 (secure 0 0 0) 0x7facd4028440 con 0x7face805fd70 2022-11-27T15:01:43.114 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.111+0000 7faceec3f700 1 -- 172.21.15.39:0/2655706375 >> v1:172.21.15.39:6816/104537 conn(0x7facc8038dd0 legacy=0x7facc803b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.115 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.111+0000 7faceec3f700 1 -- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face805fd70 msgr2=0x7face80a2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.111+0000 7faceec3f700 1 --2- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face805fd70 0x7face80a2650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7facd4002f70 tx=0x7facd4015000).stop 2022-11-27T15:01:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.111+0000 7faceec3f700 1 -- 172.21.15.39:0/2655706375 shutdown_connections 2022-11-27T15:01:43.117 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.111+0000 7faceec3f700 1 --2- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7face8133350 0x7face8145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.117 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.111+0000 7faceec3f700 1 --2- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face805d5d0 0x7face80a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.117 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.111+0000 7faceec3f700 1 --2- 172.21.15.39:0/2655706375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face805fd70 0x7face80a2650 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.118 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.111+0000 7faceec3f700 1 -- 172.21.15.39:0/2655706375 >> 172.21.15.39:0/2655706375 conn(0x7face8066e50 msgr2=0x7face81373b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:43.119 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.111+0000 7faceec3f700 1 -- 172.21.15.39:0/2655706375 shutdown_connections 2022-11-27T15:01:43.119 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.111+0000 7faceec3f700 1 -- 172.21.15.39:0/2655706375 wait complete. 2022-11-27T15:01:43.120 INFO:teuthology.orchestra.run.smithi039.stderr:enabled application 'rbd' on pool 'rbd' 2022-11-27T15:01:43.130 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-11-27T15:01:43.131 INFO:tasks.ceph.mds.a:Restarting daemon 2022-11-27T15:01:43.131 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-11-27T15:01:43.134 INFO:tasks.ceph.mds.a:Started 2022-11-27T15:01:43.135 INFO:tasks.ceph.mds.b:Restarting daemon 2022-11-27T15:01:43.135 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-11-27T15:01:43.140 INFO:tasks.ceph.mds.b:Started 2022-11-27T15:01:43.141 INFO:tasks.ceph.mds.c:Restarting daemon 2022-11-27T15:01:43.142 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-11-27T15:01:43.145 INFO:tasks.ceph.mds.c:Started 2022-11-27T15:01:43.146 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-11-27T15:01:43.147 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T15:01:43.223 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2022-11-27T15:01:43.224 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2022-11-27T15:01:43.229 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2022-11-27T15:01:43.279 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.277+0000 7fcda17f9700 1 Processor -- start 2022-11-27T15:01:43.279 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.277+0000 7fcda17f9700 1 -- start start 2022-11-27T15:01:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcda17f9700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1335b0 0x7fcd9c0641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcda17f9700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c13b600 0x7fcd9c13d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcda17f9700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c132e80 con 0x7fcd9c133990 2022-11-27T15:01:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcda17f9700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0629d0 con 0x7fcd9c1335b0 2022-11-27T15:01:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcda17f9700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0639e0 con 0x7fcd9c13b600 2022-11-27T15:01:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcd9a7fc700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcd9c133990 0x7fcd9c0622c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56852/0 (socket says 172.21.15.39:56852) 2022-11-27T15:01:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcd9a7fc700 1 -- 172.21.15.39:0/3188325663 learned_addr learned my addr 172.21.15.39:0/3188325663 (peer_addr_for_me v1:172.21.15.39:0/0) 2022-11-27T15:01:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcd9b7fe700 1 --2- 172.21.15.39:0/3188325663 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c13b600 0x7fcd9c13d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcd9affd700 1 --2- 172.21.15.39:0/3188325663 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1335b0 0x7fcd9c0641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcd99ffb700 1 -- 172.21.15.39:0/3188325663 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640906247 0 0) 0x7fcd9c132e80 con 0x7fcd9c133990 2022-11-27T15:01:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcd9b7fe700 1 -- 172.21.15.39:0/3188325663 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1335b0 msgr2=0x7fcd9c0641c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcd9b7fe700 1 --2- 172.21.15.39:0/3188325663 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1335b0 0x7fcd9c0641c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcd9b7fe700 1 -- 172.21.15.39:0/3188325663 >> v1:172.21.15.39:6789/0 conn(0x7fcd9c133990 legacy=0x7fcd9c0622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcd9b7fe700 1 -- 172.21.15.39:0/3188325663 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c063360 con 0x7fcd9c13b600 2022-11-27T15:01:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.278+0000 7fcd9affd700 1 --2- 172.21.15.39:0/3188325663 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1335b0 0x7fcd9c0641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcd9b7fe700 1 --2- 172.21.15.39:0/3188325663 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c13b600 0x7fcd9c13d9f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fcd8c002f70 tx=0x7fcd8c006df0).ready entity=mon.2 client_cookie=e6bdbdaf4aed2067 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcd99ffb700 1 -- 172.21.15.39:0/3188325663 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd8c014070 con 0x7fcd9c13b600 2022-11-27T15:01:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcd99ffb700 1 -- 172.21.15.39:0/3188325663 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c000e90 con 0x7fcd9c13b600 2022-11-27T15:01:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcd99ffb700 1 -- 172.21.15.39:0/3188325663 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd8c00fb60 con 0x7fcd9c13b600 2022-11-27T15:01:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcda17f9700 1 -- 172.21.15.39:0/3188325663 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c13b600 msgr2=0x7fcd9c13d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcda17f9700 1 --2- 172.21.15.39:0/3188325663 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c13b600 0x7fcd9c13d9f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fcd8c002f70 tx=0x7fcd8c006df0).stop 2022-11-27T15:01:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcda17f9700 1 -- 172.21.15.39:0/3188325663 shutdown_connections 2022-11-27T15:01:43.291 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcda17f9700 1 --2- 172.21.15.39:0/3188325663 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c13b600 0x7fcd9c13d9f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.291 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcda17f9700 1 --2- 172.21.15.39:0/3188325663 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1335b0 0x7fcd9c0641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.292 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcda17f9700 1 -- 172.21.15.39:0/3188325663 >> 172.21.15.39:0/3188325663 conn(0x7fcd9c06c410 msgr2=0x7fcd9c05ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:43.292 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcda17f9700 1 -- 172.21.15.39:0/3188325663 shutdown_connections 2022-11-27T15:01:43.292 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.279+0000 7fcda17f9700 1 -- 172.21.15.39:0/3188325663 wait complete. 2022-11-27T15:01:43.292 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.280+0000 7fcda17f9700 1 Processor -- start 2022-11-27T15:01:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.280+0000 7fcda17f9700 1 -- start start 2022-11-27T15:01:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcda17f9700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c1335b0 0x7fcd9c098530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcda17f9700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c133990 0x7fcd9c0979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcda17f9700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c13b600 0x7fcd9c148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcda17f9700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1353e0 con 0x7fcd9c133990 2022-11-27T15:01:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcda17f9700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1355f0 con 0x7fcd9c13b600 2022-11-27T15:01:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcda17f9700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c098a20 con 0x7fcd9c1335b0 2022-11-27T15:01:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c133990 0x7fcd9c0979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9b7fe700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c13b600 0x7fcd9c148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9affd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c1335b0 0x7fcd9c098530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9a7fc700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c133990 0x7fcd9c0979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38326/0 (socket says 172.21.15.39:38326) 2022-11-27T15:01:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9a7fc700 1 -- 172.21.15.39:0/1832490142 learned_addr learned my addr 172.21.15.39:0/1832490142 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9b7fe700 1 -- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c1335b0 msgr2=0x7fcd9c098530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9b7fe700 1 --2- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c1335b0 0x7fcd9c098530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9b7fe700 1 -- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c133990 msgr2=0x7fcd9c0979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9b7fe700 1 --2- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c133990 0x7fcd9c0979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9b7fe700 1 -- 172.21.15.39:0/1832490142 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd94009470 con 0x7fcd9c13b600 2022-11-27T15:01:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9affd700 1 --2- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c1335b0 0x7fcd9c098530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd9b7fe700 1 --2- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c13b600 0x7fcd9c148e70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fcd8c002f70 tx=0x7fcd8c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.281+0000 7fcd8bfff700 1 -- 172.21.15.39:0/1832490142 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd8c00e1d0 con 0x7fcd9c13b600 2022-11-27T15:01:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.282+0000 7fcd8bfff700 1 -- 172.21.15.39:0/1832490142 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c006a60 con 0x7fcd9c13b600 2022-11-27T15:01:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.282+0000 7fcd8bfff700 1 -- 172.21.15.39:0/1832490142 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd8c010de0 con 0x7fcd9c13b600 2022-11-27T15:01:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.282+0000 7fcda17f9700 1 -- 172.21.15.39:0/1832490142 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c00b180 con 0x7fcd9c13b600 2022-11-27T15:01:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.282+0000 7fcda17f9700 1 -- 172.21.15.39:0/1832490142 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c096b60 con 0x7fcd9c13b600 2022-11-27T15:01:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.282+0000 7fcd8bfff700 1 -- 172.21.15.39:0/1832490142 <== mon.1 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fcd8c022020 con 0x7fcd9c13b600 2022-11-27T15:01:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.282+0000 7fcda17f9700 1 -- 172.21.15.39:0/1832490142 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c057a60 con 0x7fcd9c13b600 2022-11-27T15:01:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.283+0000 7fcd8bfff700 1 -- 172.21.15.39:0/1832490142 <== mon.1 v2:172.21.15.39:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fcd8c04bdc0 con 0x7fcd9c13b600 2022-11-27T15:01:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.286+0000 7fcd8bfff700 1 -- 172.21.15.39:0/1832490142 <== mon.1 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fcd8c025e60 con 0x7fcd9c13b600 2022-11-27T15:01:43.404 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.403+0000 7fcda17f9700 1 -- 172.21.15.39:0/1832490142 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd9c064bb0 con 0x7fcd9c13b600 2022-11-27T15:01:43.405 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.403+0000 7fcd8bfff700 1 -- 172.21.15.39:0/1832490142 <== mon.1 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7657 (secure 0 0 0) 0x7fcd8c025400 con 0x7fcd9c13b600 2022-11-27T15:01:43.405 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:43.406 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"fsid":"fb73c991-5ff5-4983-821e-38ecbf33964f","created":"2022-11-27T15:01:32.796785+0000","modified":"2022-11-27T15:01:43.095176+0000","last_up_change":"2022-11-27T15:01:38.820289+0000","last_in_change":"2022-11-27T15:01:38.820289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-11-27T15:01:34.849418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T15:01:41.877873+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba0fb473-e933-4ad1-a5f8-2cdd4138f3ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6800","nonce":105137}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6801","nonce":105137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6803","nonce":105137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6802","nonce":105137}]},"public_addr":"172.21.15.39:6800/105137","cluster_addr":"172.21.15.39:6801/105137","heartbeat_back_addr":"172.21.15.39:6803/105137","heartbeat_front_addr":"172.21.15.39:6802/105137","state":["exists","up"]},{"osd":1,"uuid":"d3e5e074-117b-4420-8340-e241cb3e6553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6805","nonce":105140}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6809","nonce":105140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6811","nonce":105140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6810","nonce":105140}]},"public_addr":"172.21.15.39:6805/105140","cluster_addr":"172.21.15.39:6809/105140","heartbeat_back_addr":"172.21.15.39:6811/105140","heartbeat_front_addr":"172.21.15.39:6810/105140","state":["exists","up"]},{"osd":2,"uuid":"2d655d83-8b81-4419-a44f-f4833633c621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6812","nonce":105139}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6813","nonce":105139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6815","nonce":105139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6814","nonce":105139}]},"public_addr":"172.21.15.39:6812/105139","cluster_addr":"172.21.15.39:6813/105139","heartbeat_back_addr":"172.21.15.39:6815/105139","heartbeat_front_addr":"172.21.15.39:6814/105139","state":["exists","up"]},{"osd":3,"uuid":"1f4c9a3a-a40b-43ae-8bd1-7672f6d9a51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6804","nonce":105138}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6806","nonce":105138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6808","nonce":105138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6807","nonce":105138}]},"public_addr":"172.21.15.39:6804/105138","cluster_addr":"172.21.15.39:6806/105138","heartbeat_back_addr":"172.21.15.39:6808/105138","heartbeat_front_addr":"172.21.15.39:6807/105138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.532356+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.825983+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.875364+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.883149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T15:01:43.407 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.405+0000 7fcda17f9700 1 -- 172.21.15.39:0/1832490142 >> v1:172.21.15.39:6816/104537 conn(0x7fcd78038da0 legacy=0x7fcd7803b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.408 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.405+0000 7fcda17f9700 1 -- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c13b600 msgr2=0x7fcd9c148e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.408 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.405+0000 7fcda17f9700 1 --2- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c13b600 0x7fcd9c148e70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fcd8c002f70 tx=0x7fcd8c016000).stop 2022-11-27T15:01:43.408 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.405+0000 7fcda17f9700 1 -- 172.21.15.39:0/1832490142 shutdown_connections 2022-11-27T15:01:43.408 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.405+0000 7fcda17f9700 1 --2- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c13b600 0x7fcd9c148e70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.409 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.405+0000 7fcda17f9700 1 --2- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fcd9c1335b0 0x7fcd9c098530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.409 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.405+0000 7fcda17f9700 1 --2- 172.21.15.39:0/1832490142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c133990 0x7fcd9c0979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.409 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.405+0000 7fcda17f9700 1 -- 172.21.15.39:0/1832490142 >> 172.21.15.39:0/1832490142 conn(0x7fcd9c06c410 msgr2=0x7fcd9c05c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.405+0000 7fcda17f9700 1 -- 172.21.15.39:0/1832490142 shutdown_connections 2022-11-27T15:01:43.410 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.405+0000 7fcda17f9700 1 -- 172.21.15.39:0/1832490142 wait complete. 2022-11-27T15:01:43.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-11-27T15:01:34.849418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T15:01:41.877873+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T15:01:43.419 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-11-27T15:01:43.528 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.527+0000 7f74797b4700 1 Processor -- start 2022-11-27T15:01:43.529 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.527+0000 7f74797b4700 1 -- start start 2022-11-27T15:01:43.534 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f74797b4700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7474139b70 0x7f747405be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f74797b4700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f747405ccd0 0x7f7474139f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f74797b4700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7474139620 con 0x7f747405d6c0 2022-11-27T15:01:43.536 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f74797b4700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747413a440 con 0x7f747405ccd0 2022-11-27T15:01:43.536 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f74797b4700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f747405d110 con 0x7f7474139b70 2022-11-27T15:01:43.536 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7472ffd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7474139b70 0x7f747405be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.537 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7472ffd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7474139b70 0x7f747405be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:45558/0 (socket says 172.21.15.39:45558) 2022-11-27T15:01:43.538 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f74737fe700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f747405d6c0 0x7f747405fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56860/0 (socket says 172.21.15.39:56860) 2022-11-27T15:01:43.538 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7472ffd700 1 -- 172.21.15.39:0/3183025388 learned_addr learned my addr 172.21.15.39:0/3183025388 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:43.539 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f74727fc700 1 --2- 172.21.15.39:0/3183025388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f747405ccd0 0x7f7474139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.539 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7472ffd700 1 -- 172.21.15.39:0/3183025388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f747405ccd0 msgr2=0x7f7474139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:43.540 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7472ffd700 1 --2- 172.21.15.39:0/3183025388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f747405ccd0 0x7f7474139f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.540 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7472ffd700 1 -- 172.21.15.39:0/3183025388 >> v1:172.21.15.39:6789/0 conn(0x7f747405d6c0 legacy=0x7f747405fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.541 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7471ffb700 1 -- 172.21.15.39:0/3183025388 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317431572 0 0) 0x7f7474139620 con 0x7f747405d6c0 2022-11-27T15:01:43.541 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7472ffd700 1 -- 172.21.15.39:0/3183025388 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747405c4f0 con 0x7f7474139b70 2022-11-27T15:01:43.541 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7472ffd700 1 --2- 172.21.15.39:0/3183025388 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7474139b70 0x7f747405be80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7464002f70 tx=0x7f7464006eb0).ready entity=mon.2 client_cookie=8ba2efbdbdc9a00d server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:43.542 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7471ffb700 1 -- 172.21.15.39:0/3183025388 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f746400d570 con 0x7f7474139b70 2022-11-27T15:01:43.542 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7471ffb700 1 -- 172.21.15.39:0/3183025388 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7464000e90 con 0x7f7474139b70 2022-11-27T15:01:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.528+0000 7f7471ffb700 1 -- 172.21.15.39:0/3183025388 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f746400d8c0 con 0x7f7474139b70 2022-11-27T15:01:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.529+0000 7f74797b4700 1 -- 172.21.15.39:0/3183025388 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7474139b70 msgr2=0x7f747405be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.529+0000 7f74797b4700 1 --2- 172.21.15.39:0/3183025388 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7474139b70 0x7f747405be80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7464002f70 tx=0x7f7464006eb0).stop 2022-11-27T15:01:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.529+0000 7f74797b4700 1 -- 172.21.15.39:0/3183025388 shutdown_connections 2022-11-27T15:01:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.529+0000 7f74797b4700 1 --2- 172.21.15.39:0/3183025388 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7474139b70 0x7f747405be80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.545 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.529+0000 7f74797b4700 1 --2- 172.21.15.39:0/3183025388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f747405ccd0 0x7f7474139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.545 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.529+0000 7f74797b4700 1 -- 172.21.15.39:0/3183025388 >> 172.21.15.39:0/3183025388 conn(0x7f747406c640 msgr2=0x7f74740ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:43.546 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.529+0000 7f74797b4700 1 -- 172.21.15.39:0/3183025388 shutdown_connections 2022-11-27T15:01:43.546 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.529+0000 7f74797b4700 1 -- 172.21.15.39:0/3183025388 wait complete. 2022-11-27T15:01:43.547 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f74797b4700 1 Processor -- start 2022-11-27T15:01:43.548 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f74797b4700 1 -- start start 2022-11-27T15:01:43.548 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f74797b4700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f747405ccd0 0x7f7474116f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.548 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f74797b4700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f747405d6c0 0x7f7474115a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.549 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f74797b4700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7474139b70 0x7f7474145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.549 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f74797b4700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7474116a00 con 0x7f7474139b70 2022-11-27T15:01:43.549 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f74797b4700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7474116790 con 0x7f747405d6c0 2022-11-27T15:01:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f74797b4700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7474145530 con 0x7f747405ccd0 2022-11-27T15:01:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f7472ffd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f747405ccd0 0x7f7474116f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f74737fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7474139b70 0x7f7474145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.551 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f7472ffd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f747405ccd0 0x7f7474116f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:45562/0 (socket says 172.21.15.39:45562) 2022-11-27T15:01:43.553 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.530+0000 7f7472ffd700 1 -- 172.21.15.39:0/255156988 learned_addr learned my addr 172.21.15.39:0/255156988 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:43.553 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f74727fc700 1 --2- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f747405d6c0 0x7f7474115a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.554 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f7472ffd700 1 -- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f747405d6c0 msgr2=0x7f7474115a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.554 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f7472ffd700 1 --2- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f747405d6c0 0x7f7474115a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.555 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f7472ffd700 1 -- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7474139b70 msgr2=0x7f7474145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.555 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f7472ffd700 1 --2- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7474139b70 0x7f7474145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.561 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f7472ffd700 1 -- 172.21.15.39:0/255156988 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746400b1e0 con 0x7f747405ccd0 2022-11-27T15:01:43.562 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f7472ffd700 1 --2- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f747405ccd0 0x7f7474116f00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f7464002f70 tx=0x7f7464015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:43.562 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f7463fff700 1 -- 172.21.15.39:0/255156988 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74640068b0 con 0x7f747405ccd0 2022-11-27T15:01:43.562 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f7463fff700 1 -- 172.21.15.39:0/255156988 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746400d8f0 con 0x7f747405ccd0 2022-11-27T15:01:43.563 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f74797b4700 1 -- 172.21.15.39:0/255156988 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74741457d0 con 0x7f747405ccd0 2022-11-27T15:01:43.563 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f7463fff700 1 -- 172.21.15.39:0/255156988 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7464011e20 con 0x7f747405ccd0 2022-11-27T15:01:43.564 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.531+0000 7f74797b4700 1 -- 172.21.15.39:0/255156988 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74740a6240 con 0x7f747405ccd0 2022-11-27T15:01:43.564 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.532+0000 7f7463fff700 1 -- 172.21.15.39:0/255156988 <== mon.2 v2:172.21.15.39:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f746400f070 con 0x7f747405ccd0 2022-11-27T15:01:43.564 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.532+0000 7f74797b4700 1 -- 172.21.15.39:0/255156988 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747405db70 con 0x7f747405ccd0 2022-11-27T15:01:43.566 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.532+0000 7f7463fff700 1 -- 172.21.15.39:0/255156988 <== mon.2 v2:172.21.15.39:3302/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f746404bf90 con 0x7f747405ccd0 2022-11-27T15:01:43.566 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.535+0000 7f7463fff700 1 -- 172.21.15.39:0/255156988 <== mon.2 v2:172.21.15.39:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f74640295b0 con 0x7f747405ccd0 2022-11-27T15:01:43.655 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.653+0000 7f74797b4700 1 -- 172.21.15.39:0/255156988 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7474057a60 con 0x7f747405ccd0 2022-11-27T15:01:43.656 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.654+0000 7f7463fff700 1 -- 172.21.15.39:0/255156988 <== mon.2 v2:172.21.15.39:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v14) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7464007ce0 con 0x7f747405ccd0 2022-11-27T15:01:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2022-11-27T15:01:43.658 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.655+0000 7f74797b4700 1 -- 172.21.15.39:0/255156988 >> v1:172.21.15.39:6816/104537 conn(0x7f7450038ec0 legacy=0x7f745003b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.658 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.655+0000 7f74797b4700 1 -- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f747405ccd0 msgr2=0x7f7474116f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.659 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.655+0000 7f74797b4700 1 --2- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f747405ccd0 0x7f7474116f00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f7464002f70 tx=0x7f7464015000).stop 2022-11-27T15:01:43.659 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.655+0000 7f74797b4700 1 -- 172.21.15.39:0/255156988 shutdown_connections 2022-11-27T15:01:43.659 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.655+0000 7f74797b4700 1 --2- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7474139b70 0x7f7474145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.659 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.655+0000 7f74797b4700 1 --2- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f747405ccd0 0x7f7474116f00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.660 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.655+0000 7f74797b4700 1 --2- 172.21.15.39:0/255156988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f747405d6c0 0x7f7474115a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.660 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.655+0000 7f74797b4700 1 -- 172.21.15.39:0/255156988 >> 172.21.15.39:0/255156988 conn(0x7f747406c640 msgr2=0x7f7474138590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:43.660 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.655+0000 7f74797b4700 1 -- 172.21.15.39:0/255156988 shutdown_connections 2022-11-27T15:01:43.660 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.655+0000 7f74797b4700 1 -- 172.21.15.39:0/255156988 wait complete. 2022-11-27T15:01:43.673 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T15:01:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.781+0000 7f0bd99fd700 1 Processor -- start 2022-11-27T15:01:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.781+0000 7f0bd99fd700 1 -- start start 2022-11-27T15:01:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.782+0000 7f0bd99fd700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4099f40 0x7f0bd409b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.782+0000 7f0bd99fd700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0bd4133820 0x7f0bd4140970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.782+0000 7f0bd99fd700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd4068020 con 0x7f0bd409ba10 2022-11-27T15:01:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.782+0000 7f0bd99fd700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd40682c0 con 0x7f0bd4133820 2022-11-27T15:01:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.782+0000 7f0bd99fd700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4067360 con 0x7f0bd4099f40 2022-11-27T15:01:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.782+0000 7f0bd2ffd700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4099f40 0x7f0bd409b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.782+0000 7f0bd2ffd700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4099f40 0x7f0bd409b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:56406/0 (socket says 172.21.15.39:56406) 2022-11-27T15:01:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.782+0000 7f0bd2ffd700 1 -- 172.21.15.39:0/1328330219 learned_addr learned my addr 172.21.15.39:0/1328330219 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.782+0000 7f0bd37fe700 1 --2- 172.21.15.39:0/1328330219 >> v2:172.21.15.39:3300/0 conn(0x7f0bd4133820 0x7f0bd4140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.797 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd37fe700 1 -- 172.21.15.39:0/1328330219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4099f40 msgr2=0x7f0bd409b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:43.797 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd37fe700 1 --2- 172.21.15.39:0/1328330219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4099f40 0x7f0bd409b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.797 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd37fe700 1 -- 172.21.15.39:0/1328330219 >> v1:172.21.15.39:6789/0 conn(0x7f0bd409ba10 legacy=0x7f0bd4133110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.797 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd37fe700 1 -- 172.21.15.39:0/1328330219 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd405d6a0 con 0x7f0bd4133820 2022-11-27T15:01:43.797 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd2ffd700 1 --2- 172.21.15.39:0/1328330219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4099f40 0x7f0bd409b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd37fe700 1 --2- 172.21.15.39:0/1328330219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4133820 0x7f0bd4140970 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f0bcc0032f0 tx=0x7f0bcc0091d0).ready entity=mon.0 client_cookie=79cb11929eb312ab server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd1ffb700 1 -- 172.21.15.39:0/1328330219 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bcc00b6c0 con 0x7f0bd4133820 2022-11-27T15:01:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd1ffb700 1 -- 172.21.15.39:0/1328330219 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc00b820 con 0x7f0bd4133820 2022-11-27T15:01:43.798 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd1ffb700 1 -- 172.21.15.39:0/1328330219 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bcc00b990 con 0x7f0bd4133820 2022-11-27T15:01:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1328330219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4133820 msgr2=0x7f0bd4140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.783+0000 7f0bd99fd700 1 --2- 172.21.15.39:0/1328330219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4133820 0x7f0bd4140970 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f0bcc0032f0 tx=0x7f0bcc0091d0).stop 2022-11-27T15:01:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.784+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1328330219 shutdown_connections 2022-11-27T15:01:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.784+0000 7f0bd99fd700 1 --2- 172.21.15.39:0/1328330219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4099f40 0x7f0bd409b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.784+0000 7f0bd99fd700 1 --2- 172.21.15.39:0/1328330219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4133820 0x7f0bd4140970 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.784+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1328330219 >> 172.21.15.39:0/1328330219 conn(0x7f0bd406c410 msgr2=0x7f0bd405ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.784+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1328330219 shutdown_connections 2022-11-27T15:01:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.784+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1328330219 wait complete. 2022-11-27T15:01:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.784+0000 7f0bd99fd700 1 Processor -- start 2022-11-27T15:01:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.784+0000 7f0bd99fd700 1 -- start start 2022-11-27T15:01:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd99fd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4099f40 0x7f0bd4094790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd99fd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f0bd409ba10 0x7f0bd4148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd99fd700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4133820 0x7f0bd4149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd99fd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd41459f0 con 0x7f0bd4099f40 2022-11-27T15:01:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd99fd700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4096430 con 0x7f0bd4133820 2022-11-27T15:01:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd99fd700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4095160 con 0x7f0bd409ba10 2022-11-27T15:01:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4099f40 0x7f0bd4094790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4099f40 0x7f0bd4094790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38350/0 (socket says 172.21.15.39:38350) 2022-11-27T15:01:43.803 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd2ffd700 1 -- 172.21.15.39:0/1283515340 learned_addr learned my addr 172.21.15.39:0/1283515340 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:43.803 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd37fe700 1 --2- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4133820 0x7f0bd4149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.804 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd27fc700 1 --2- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f0bd409ba10 0x7f0bd4148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:43.804 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd2ffd700 1 -- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f0bd409ba10 msgr2=0x7f0bd4148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.804 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd2ffd700 1 --2- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f0bd409ba10 0x7f0bd4148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.805 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd2ffd700 1 -- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4133820 msgr2=0x7f0bd4149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.805 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd2ffd700 1 --2- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4133820 0x7f0bd4149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.806 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.785+0000 7f0bd2ffd700 1 -- 172.21.15.39:0/1283515340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc400b1e0 con 0x7f0bd4099f40 2022-11-27T15:01:43.806 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.786+0000 7f0bd2ffd700 1 --2- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4099f40 0x7f0bd4094790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f0bc400c000 tx=0x7f0bc4006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:43.806 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.786+0000 7f0bcbfff700 1 -- 172.21.15.39:0/1283515340 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bc4012600 con 0x7f0bd4099f40 2022-11-27T15:01:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.786+0000 7f0bcbfff700 1 -- 172.21.15.39:0/1283515340 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc4012760 con 0x7f0bd4099f40 2022-11-27T15:01:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.786+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1283515340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc009470 con 0x7f0bd4099f40 2022-11-27T15:01:43.808 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.786+0000 7f0bcbfff700 1 -- 172.21.15.39:0/1283515340 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bc4012ac0 con 0x7f0bd4099f40 2022-11-27T15:01:43.808 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.786+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1283515340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd4094260 con 0x7f0bd4099f40 2022-11-27T15:01:43.808 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.787+0000 7f0bcbfff700 1 -- 172.21.15.39:0/1283515340 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f0bc4022070 con 0x7f0bd4099f40 2022-11-27T15:01:43.808 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.787+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1283515340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd4099900 con 0x7f0bd4099f40 2022-11-27T15:01:43.809 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.787+0000 7f0bcbfff700 1 -- 172.21.15.39:0/1283515340 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f0bc404a8f0 con 0x7f0bd4099f40 2022-11-27T15:01:43.809 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.790+0000 7f0bcbfff700 1 -- 172.21.15.39:0/1283515340 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f0bc4033570 con 0x7f0bd4099f40 2022-11-27T15:01:43.910 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.907+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1283515340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0bd4095b40 con 0x7f0bd4099f40 2022-11-27T15:01:43.910 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.908+0000 7f0bcbfff700 1 -- 172.21.15.39:0/1283515340 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v14) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0bc4033160 con 0x7f0bd4099f40 2022-11-27T15:01:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 8 2022-11-27T15:01:43.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.909+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1283515340 >> v1:172.21.15.39:6816/104537 conn(0x7f0bb8038dd0 legacy=0x7f0bb803b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.909+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4099f40 msgr2=0x7f0bd4094790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:43.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.909+0000 7f0bd99fd700 1 --2- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4099f40 0x7f0bd4094790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f0bc400c000 tx=0x7f0bc4006eb0).stop 2022-11-27T15:01:43.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.909+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1283515340 shutdown_connections 2022-11-27T15:01:43.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.909+0000 7f0bd99fd700 1 --2- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bd4099f40 0x7f0bd4094790 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.909+0000 7f0bd99fd700 1 --2- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bd4133820 0x7f0bd4149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.909+0000 7f0bd99fd700 1 --2- 172.21.15.39:0/1283515340 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f0bd409ba10 0x7f0bd4148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:43.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.909+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1283515340 >> 172.21.15.39:0/1283515340 conn(0x7f0bd406c410 msgr2=0x7f0bd405ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:43.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.909+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1283515340 shutdown_connections 2022-11-27T15:01:43.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:43.909+0000 7f0bd99fd700 1 -- 172.21.15.39:0/1283515340 wait complete. 2022-11-27T15:01:43.922 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T15:01:44.029 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.027+0000 7f9f045a8700 1 Processor -- start 2022-11-27T15:01:44.029 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.027+0000 7f9f045a8700 1 -- start start 2022-11-27T15:01:44.035 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f045a8700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc099f40 0x7f9efc09b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:44.035 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f045a8700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc09ba10 0x7f9efc133110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:44.035 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f045a8700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9efc133600 0x7f9efc05b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:44.036 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f045a8700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9efc068020 con 0x7f9efc133600 2022-11-27T15:01:44.036 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f045a8700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0682c0 con 0x7f9efc09ba10 2022-11-27T15:01:44.036 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f045a8700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc067360 con 0x7f9efc099f40 2022-11-27T15:01:44.036 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f02b45700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9efc133600 0x7f9efc05b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:44.037 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f02344700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc099f40 0x7f9efc09b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:44.037 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f01b43700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc09ba10 0x7f9efc133110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:44.037 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f02344700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc099f40 0x7f9efc09b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:45586/0 (socket says 172.21.15.39:45586) 2022-11-27T15:01:44.037 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f02b45700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9efc133600 0x7f9efc05b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38366/0 (socket says 172.21.15.39:38366) 2022-11-27T15:01:44.037 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.028+0000 7f9f02344700 1 -- 172.21.15.39:0/3065029798 learned_addr learned my addr 172.21.15.39:0/3065029798 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:44.038 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f02344700 1 -- 172.21.15.39:0/3065029798 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc09ba10 msgr2=0x7f9efc133110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:44.038 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f02344700 1 --2- 172.21.15.39:0/3065029798 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc09ba10 0x7f9efc133110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:44.038 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f02344700 1 -- 172.21.15.39:0/3065029798 >> v2:172.21.15.39:3300/0 conn(0x7f9efc133600 msgr2=0x7f9efc05b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:44.038 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f02344700 1 --2- 172.21.15.39:0/3065029798 >> v2:172.21.15.39:3300/0 conn(0x7f9efc133600 0x7f9efc05b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:44.038 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f02344700 1 -- 172.21.15.39:0/3065029798 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc05d6a0 con 0x7f9efc099f40 2022-11-27T15:01:44.039 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f01b43700 1 --2- 172.21.15.39:0/3065029798 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc09ba10 0x7f9efc133110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:44.039 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f02344700 1 --2- 172.21.15.39:0/3065029798 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc099f40 0x7f9efc09b200 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f9ef8002f70 tx=0x7f9ef8006eb0).ready entity=mon.2 client_cookie=5b5f69df3cc35d6e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:44.039 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f01342700 1 -- 172.21.15.39:0/3065029798 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef800d570 con 0x7f9efc099f40 2022-11-27T15:01:44.039 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f01342700 1 -- 172.21.15.39:0/3065029798 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef8000e90 con 0x7f9efc099f40 2022-11-27T15:01:44.040 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f01342700 1 -- 172.21.15.39:0/3065029798 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef800d900 con 0x7f9efc099f40 2022-11-27T15:01:44.040 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f045a8700 1 -- 172.21.15.39:0/3065029798 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc099f40 msgr2=0x7f9efc09b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:44.040 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f045a8700 1 --2- 172.21.15.39:0/3065029798 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc099f40 0x7f9efc09b200 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f9ef8002f70 tx=0x7f9ef8006eb0).stop 2022-11-27T15:01:44.040 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f045a8700 1 -- 172.21.15.39:0/3065029798 shutdown_connections 2022-11-27T15:01:44.041 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f045a8700 1 --2- 172.21.15.39:0/3065029798 >> v2:172.21.15.39:3300/0 conn(0x7f9efc133600 0x7f9efc05b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:44.041 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f045a8700 1 --2- 172.21.15.39:0/3065029798 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc099f40 0x7f9efc09b200 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:44.041 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f045a8700 1 --2- 172.21.15.39:0/3065029798 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc09ba10 0x7f9efc133110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:44.041 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f045a8700 1 -- 172.21.15.39:0/3065029798 >> 172.21.15.39:0/3065029798 conn(0x7f9efc06c410 msgr2=0x7f9efc05ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:44.041 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f045a8700 1 -- 172.21.15.39:0/3065029798 shutdown_connections 2022-11-27T15:01:44.042 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.029+0000 7f9f045a8700 1 -- 172.21.15.39:0/3065029798 wait complete. 2022-11-27T15:01:44.042 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.030+0000 7f9f045a8700 1 Processor -- start 2022-11-27T15:01:44.042 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.030+0000 7f9f045a8700 1 -- start start 2022-11-27T15:01:44.042 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f045a8700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc099f40 0x7f9efc093b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:44.043 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f045a8700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc09ba10 0x7f9efc1489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:44.043 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f045a8700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9efc133600 0x7f9efc1350c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:44.043 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f045a8700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc094050 con 0x7f9efc133600 2022-11-27T15:01:44.043 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f045a8700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc149360 con 0x7f9efc099f40 2022-11-27T15:01:44.044 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f045a8700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc149620 con 0x7f9efc09ba10 2022-11-27T15:01:44.044 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f01b43700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc09ba10 0x7f9efc1489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:44.044 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f02b45700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9efc133600 0x7f9efc1350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:44.044 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f01b43700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc09ba10 0x7f9efc1489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:45590/0 (socket says 172.21.15.39:45590) 2022-11-27T15:01:44.044 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f01b43700 1 -- 172.21.15.39:0/1023690638 learned_addr learned my addr 172.21.15.39:0/1023690638 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:44.045 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f02344700 1 --2- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc099f40 0x7f9efc093b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:44.045 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f02b45700 1 -- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc09ba10 msgr2=0x7f9efc1489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:44.045 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f02b45700 1 --2- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc09ba10 0x7f9efc1489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:44.045 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f02b45700 1 -- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc099f40 msgr2=0x7f9efc093b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:44.046 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f02b45700 1 --2- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc099f40 0x7f9efc093b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:44.046 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f02b45700 1 -- 172.21.15.39:0/1023690638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef000b180 con 0x7f9efc133600 2022-11-27T15:01:44.046 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f02344700 1 --2- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc099f40 0x7f9efc093b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:44.046 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.031+0000 7f9f01b43700 1 --2- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc09ba10 0x7f9efc1489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:44.046 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.032+0000 7f9f02b45700 1 --2- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9efc133600 0x7f9efc1350c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9ef400c000 tx=0x7f9ef40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.032+0000 7f9eef7fe700 1 -- 172.21.15.39:0/1023690638 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef4012600 con 0x7f9efc133600 2022-11-27T15:01:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.032+0000 7f9eef7fe700 1 -- 172.21.15.39:0/1023690638 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef4012760 con 0x7f9efc133600 2022-11-27T15:01:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.032+0000 7f9f045a8700 1 -- 172.21.15.39:0/1023690638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef800b1e0 con 0x7f9efc133600 2022-11-27T15:01:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.032+0000 7f9eef7fe700 1 -- 172.21.15.39:0/1023690638 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ef4012ac0 con 0x7f9efc133600 2022-11-27T15:01:44.048 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.032+0000 7f9f045a8700 1 -- 172.21.15.39:0/1023690638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc13d970 con 0x7f9efc133600 2022-11-27T15:01:44.048 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.033+0000 7f9eef7fe700 1 -- 172.21.15.39:0/1023690638 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f9ef4022070 con 0x7f9efc133600 2022-11-27T15:01:44.048 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.033+0000 7f9f045a8700 1 -- 172.21.15.39:0/1023690638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9efc099900 con 0x7f9efc133600 2022-11-27T15:01:44.048 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.033+0000 7f9eef7fe700 1 -- 172.21.15.39:0/1023690638 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f9ef404aa30 con 0x7f9efc133600 2022-11-27T15:01:44.049 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.036+0000 7f9eef7fe700 1 -- 172.21.15.39:0/1023690638 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f9ef4016070 con 0x7f9efc133600 2022-11-27T15:01:44.189 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.187+0000 7f9f045a8700 1 -- 172.21.15.39:0/1023690638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9efc057a60 con 0x7f9efc133600 2022-11-27T15:01:44.189 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.188+0000 7f9eef7fe700 1 -- 172.21.15.39:0/1023690638 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+794 (secure 0 0 0) 0x7f9ef4016020 con 0x7f9efc133600 2022-11-27T15:01:44.190 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:44.190 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4194,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6818/2768721463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6818","nonce":2768721463}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[]} 2022-11-27T15:01:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.189+0000 7f9f045a8700 1 -- 172.21.15.39:0/1023690638 >> v1:172.21.15.39:6816/104537 conn(0x7f9edc038da0 legacy=0x7f9edc03b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.189+0000 7f9f045a8700 1 -- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9efc133600 msgr2=0x7f9efc1350c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.189+0000 7f9f045a8700 1 --2- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9efc133600 0x7f9efc1350c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9ef400c000 tx=0x7f9ef40091d0).stop 2022-11-27T15:01:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.189+0000 7f9f045a8700 1 -- 172.21.15.39:0/1023690638 shutdown_connections 2022-11-27T15:01:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.189+0000 7f9f045a8700 1 --2- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9efc133600 0x7f9efc1350c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.189+0000 7f9f045a8700 1 --2- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9efc099f40 0x7f9efc093b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.189+0000 7f9f045a8700 1 --2- 172.21.15.39:0/1023690638 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f9efc09ba10 0x7f9efc1489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.189+0000 7f9f045a8700 1 -- 172.21.15.39:0/1023690638 >> 172.21.15.39:0/1023690638 conn(0x7f9efc06c410 msgr2=0x7f9efc05ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.189+0000 7f9f045a8700 1 -- 172.21.15.39:0/1023690638 shutdown_connections 2022-11-27T15:01:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:44.189+0000 7f9f045a8700 1 -- 172.21.15.39:0/1023690638 wait complete. 2022-11-27T15:01:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 2 2022-11-27T15:01:46.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T15:01:46.330 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.321+0000 7fc1525cd700 1 Processor -- start 2022-11-27T15:01:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.321+0000 7fc1525cd700 1 -- start start 2022-11-27T15:01:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.322+0000 7fc1525cd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c05be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.322+0000 7fc1525cd700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc14c05ccd0 0x7fc14c139f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.322+0000 7fc1525cd700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc14c139620 con 0x7fc14c05d6c0 2022-11-27T15:01:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.322+0000 7fc1525cd700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc14c13a440 con 0x7fc14c05ccd0 2022-11-27T15:01:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.322+0000 7fc1525cd700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c05d110 con 0x7fc14c139b70 2022-11-27T15:01:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.322+0000 7fc14b7fe700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc14c05ccd0 0x7fc14c139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:46.332 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.322+0000 7fc14bfff700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c05be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.322+0000 7fc150b6a700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc14c05d6c0 0x7fc14c05fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:56882/0 (socket says 172.21.15.39:56882) 2022-11-27T15:01:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.322+0000 7fc150b6a700 1 -- 172.21.15.39:0/4035737790 learned_addr learned my addr 172.21.15.39:0/4035737790 (peer_addr_for_me v1:172.21.15.39:0/0) 2022-11-27T15:01:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc14b7fe700 1 -- 172.21.15.39:0/4035737790 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 msgr2=0x7fc14c05be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc14b7fe700 1 --2- 172.21.15.39:0/4035737790 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c05be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:46.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc14b7fe700 1 -- 172.21.15.39:0/4035737790 >> v1:172.21.15.39:6789/0 conn(0x7fc14c05d6c0 legacy=0x7fc14c05fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:46.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc14affd700 1 -- 172.21.15.39:0/4035737790 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443835019 0 0) 0x7fc14c139620 con 0x7fc14c05d6c0 2022-11-27T15:01:46.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc14b7fe700 1 -- 172.21.15.39:0/4035737790 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c05c4f0 con 0x7fc14c05ccd0 2022-11-27T15:01:46.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc14bfff700 1 --2- 172.21.15.39:0/4035737790 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c05be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:46.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc14b7fe700 1 --2- 172.21.15.39:0/4035737790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 0x7fc14c139f50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fc140002f70 tx=0x7fc140006df0).ready entity=mon.0 client_cookie=216859848df9535e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:46.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc14affd700 1 -- 172.21.15.39:0/4035737790 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc14000d570 con 0x7fc14c05ccd0 2022-11-27T15:01:46.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc14affd700 1 -- 172.21.15.39:0/4035737790 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc140000e90 con 0x7fc14c05ccd0 2022-11-27T15:01:46.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc14affd700 1 -- 172.21.15.39:0/4035737790 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc14000d8c0 con 0x7fc14c05ccd0 2022-11-27T15:01:46.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc1525cd700 1 -- 172.21.15.39:0/4035737790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 msgr2=0x7fc14c139f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:46.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc1525cd700 1 --2- 172.21.15.39:0/4035737790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 0x7fc14c139f50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fc140002f70 tx=0x7fc140006df0).stop 2022-11-27T15:01:46.336 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc1525cd700 1 -- 172.21.15.39:0/4035737790 shutdown_connections 2022-11-27T15:01:46.336 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc1525cd700 1 --2- 172.21.15.39:0/4035737790 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c05be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:46.336 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc1525cd700 1 --2- 172.21.15.39:0/4035737790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 0x7fc14c139f50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:46.336 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.323+0000 7fc1525cd700 1 -- 172.21.15.39:0/4035737790 >> 172.21.15.39:0/4035737790 conn(0x7fc14c06c640 msgr2=0x7fc14c0ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:46.337 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.324+0000 7fc1525cd700 1 -- 172.21.15.39:0/4035737790 shutdown_connections 2022-11-27T15:01:46.337 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.324+0000 7fc1525cd700 1 -- 172.21.15.39:0/4035737790 wait complete. 2022-11-27T15:01:46.337 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.324+0000 7fc1525cd700 1 Processor -- start 2022-11-27T15:01:46.337 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.324+0000 7fc1525cd700 1 -- start start 2022-11-27T15:01:46.337 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc1525cd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 0x7fc14c114d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:46.338 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc1525cd700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc14c05d6c0 0x7fc14c0a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:46.338 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc1525cd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c0a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:46.338 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc1525cd700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c116c70 con 0x7fc14c05ccd0 2022-11-27T15:01:46.338 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc1525cd700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c1141f0 con 0x7fc14c05d6c0 2022-11-27T15:01:46.339 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc1525cd700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c1169d0 con 0x7fc14c139b70 2022-11-27T15:01:46.339 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc150b6a700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c0a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:46.339 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc14bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 0x7fc14c114d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:46.340 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc14b7fe700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc14c05d6c0 0x7fc14c0a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:46.340 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc14bfff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 0x7fc14c114d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38388/0 (socket says 172.21.15.39:38388) 2022-11-27T15:01:46.340 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc150b6a700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c0a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:45602/0 (socket says 172.21.15.39:45602) 2022-11-27T15:01:46.341 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc14bfff700 1 -- 172.21.15.39:0/3953099458 learned_addr learned my addr 172.21.15.39:0/3953099458 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:46.341 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.325+0000 7fc14b7fe700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc14c05d6c0 0x7fc14c0a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:56418/0 (socket says 172.21.15.39:56418) 2022-11-27T15:01:46.341 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc14bfff700 1 -- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 msgr2=0x7fc14c0a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:46.341 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc14bfff700 1 --2- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c0a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:46.342 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc14bfff700 1 -- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc14c05d6c0 msgr2=0x7fc14c0a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:46.342 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc14bfff700 1 --2- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc14c05d6c0 0x7fc14c0a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:46.342 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc14bfff700 1 -- 172.21.15.39:0/3953099458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c00b180 con 0x7fc14c05ccd0 2022-11-27T15:01:46.342 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc14b7fe700 1 --2- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc14c05d6c0 0x7fc14c0a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:46.343 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc150b6a700 1 --2- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c0a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:46.343 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc14bfff700 1 --2- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 0x7fc14c114d60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fc14c116de0 tx=0x7fc13c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:46.343 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc1497fa700 1 -- 172.21.15.39:0/3953099458 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc13c010780 con 0x7fc14c05ccd0 2022-11-27T15:01:46.343 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc1497fa700 1 -- 172.21.15.39:0/3953099458 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c0108e0 con 0x7fc14c05ccd0 2022-11-27T15:01:46.343 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc1497fa700 1 -- 172.21.15.39:0/3953099458 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc13c010c40 con 0x7fc14c05ccd0 2022-11-27T15:01:46.344 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc1525cd700 1 -- 172.21.15.39:0/3953099458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14000b180 con 0x7fc14c05ccd0 2022-11-27T15:01:46.344 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.326+0000 7fc1525cd700 1 -- 172.21.15.39:0/3953099458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14c135de0 con 0x7fc14c05ccd0 2022-11-27T15:01:46.344 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.327+0000 7fc1497fa700 1 -- 172.21.15.39:0/3953099458 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fc13c01c460 con 0x7fc14c05ccd0 2022-11-27T15:01:46.344 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.327+0000 7fc1497fa700 1 -- 172.21.15.39:0/3953099458 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fc13c04ab10 con 0x7fc14c05ccd0 2022-11-27T15:01:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.328+0000 7fc1525cd700 1 -- 172.21.15.39:0/3953099458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14c0a6690 con 0x7fc14c05ccd0 2022-11-27T15:01:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.331+0000 7fc1497fa700 1 -- 172.21.15.39:0/3953099458 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fc13c033640 con 0x7fc14c05ccd0 2022-11-27T15:01:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.481+0000 7fc1525cd700 1 -- 172.21.15.39:0/3953099458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc14c1144d0 con 0x7fc14c05ccd0 2022-11-27T15:01:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.481+0000 7fc1497fa700 1 -- 172.21.15.39:0/3953099458 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+1084 (secure 0 0 0) 0x7fc13c02b090 con 0x7fc14c05ccd0 2022-11-27T15:01:46.484 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:46.485 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4194,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6818/2768721463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6818","nonce":2768721463}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6819/142076358","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6819","nonce":142076358}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[]} 2022-11-27T15:01:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.482+0000 7fc1525cd700 1 -- 172.21.15.39:0/3953099458 >> v1:172.21.15.39:6816/104537 conn(0x7fc12c038e60 legacy=0x7fc12c03b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.482+0000 7fc1525cd700 1 -- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 msgr2=0x7fc14c114d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.482+0000 7fc1525cd700 1 --2- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 0x7fc14c114d60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fc14c116de0 tx=0x7fc13c006df0).stop 2022-11-27T15:01:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.482+0000 7fc1525cd700 1 -- 172.21.15.39:0/3953099458 shutdown_connections 2022-11-27T15:01:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.482+0000 7fc1525cd700 1 --2- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc14c05ccd0 0x7fc14c114d60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.482+0000 7fc1525cd700 1 --2- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc14c139b70 0x7fc14c0a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.482+0000 7fc1525cd700 1 --2- 172.21.15.39:0/3953099458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc14c05d6c0 0x7fc14c0a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.482+0000 7fc1525cd700 1 -- 172.21.15.39:0/3953099458 >> 172.21.15.39:0/3953099458 conn(0x7fc14c06c640 msgr2=0x7fc14c0aa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.482+0000 7fc1525cd700 1 -- 172.21.15.39:0/3953099458 shutdown_connections 2022-11-27T15:01:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:46.482+0000 7fc1525cd700 1 -- 172.21.15.39:0/3953099458 wait complete. 2022-11-27T15:01:46.490 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 3 2022-11-27T15:01:48.495 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T15:01:48.616 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.615+0000 7fe3bfed0700 1 Processor -- start 2022-11-27T15:01:48.617 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.615+0000 7fe3bfed0700 1 -- start start 2022-11-27T15:01:48.623 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bfed0700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 0x7fe3b8139e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.624 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bfed0700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b80a1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.624 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bfed0700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe3b805b970 0x7fe3b805bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.624 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bfed0700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b80a0b60 con 0x7fe3b805b970 2022-11-27T15:01:48.625 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bfed0700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81393b0 con 0x7fe3b8139a30 2022-11-27T15:01:48.625 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bfed0700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81395e0 con 0x7fe3b80a2ea0 2022-11-27T15:01:48.625 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bd46b700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b80a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.626 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bdc6c700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 0x7fe3b8139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.626 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3be46d700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe3b805b970 0x7fe3b805bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.626 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bd46b700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b80a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:33966/0 (socket says 172.21.15.39:33966) 2022-11-27T15:01:48.626 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bd46b700 1 -- 172.21.15.39:0/554253236 learned_addr learned my addr 172.21.15.39:0/554253236 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:48.627 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bd46b700 1 -- 172.21.15.39:0/554253236 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 msgr2=0x7fe3b8139e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:48.627 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bd46b700 1 --2- 172.21.15.39:0/554253236 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 0x7fe3b8139e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.627 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bd46b700 1 -- 172.21.15.39:0/554253236 >> v2:172.21.15.39:3300/0 conn(0x7fe3b805b970 msgr2=0x7fe3b805bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:48.628 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bd46b700 1 --2- 172.21.15.39:0/554253236 >> v2:172.21.15.39:3300/0 conn(0x7fe3b805b970 0x7fe3b805bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.628 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3bd46b700 1 -- 172.21.15.39:0/554253236 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80a1080 con 0x7fe3b80a2ea0 2022-11-27T15:01:48.628 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.616+0000 7fe3be46d700 1 --2- 172.21.15.39:0/554253236 >> v2:172.21.15.39:3300/0 conn(0x7fe3b805b970 0x7fe3b805bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:48.628 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bd46b700 1 --2- 172.21.15.39:0/554253236 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b80a1860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe3b4002f70 tx=0x7fe3b4006eb0).ready entity=mon.2 client_cookie=91eee0796bed70db server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:48.629 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bcc6a700 1 -- 172.21.15.39:0/554253236 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3b400d570 con 0x7fe3b80a2ea0 2022-11-27T15:01:48.629 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bcc6a700 1 -- 172.21.15.39:0/554253236 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b4000e90 con 0x7fe3b80a2ea0 2022-11-27T15:01:48.629 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bcc6a700 1 -- 172.21.15.39:0/554253236 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3b400d900 con 0x7fe3b80a2ea0 2022-11-27T15:01:48.629 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bfed0700 1 -- 172.21.15.39:0/554253236 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 msgr2=0x7fe3b80a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:48.629 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bfed0700 1 --2- 172.21.15.39:0/554253236 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b80a1860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe3b4002f70 tx=0x7fe3b4006eb0).stop 2022-11-27T15:01:48.630 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bfed0700 1 -- 172.21.15.39:0/554253236 shutdown_connections 2022-11-27T15:01:48.630 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bfed0700 1 --2- 172.21.15.39:0/554253236 >> v2:172.21.15.39:3300/0 conn(0x7fe3b805b970 0x7fe3b805bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.630 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bfed0700 1 --2- 172.21.15.39:0/554253236 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 0x7fe3b8139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.630 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bfed0700 1 --2- 172.21.15.39:0/554253236 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b80a1860 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.631 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bfed0700 1 -- 172.21.15.39:0/554253236 >> 172.21.15.39:0/554253236 conn(0x7fe3b80a8a70 msgr2=0x7fe3b80a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:48.631 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bfed0700 1 -- 172.21.15.39:0/554253236 shutdown_connections 2022-11-27T15:01:48.631 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.617+0000 7fe3bfed0700 1 -- 172.21.15.39:0/554253236 wait complete. 2022-11-27T15:01:48.631 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.618+0000 7fe3bfed0700 1 Processor -- start 2022-11-27T15:01:48.631 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.618+0000 7fe3bfed0700 1 -- start start 2022-11-27T15:01:48.632 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bfed0700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3b805b970 0x7fe3b8094200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.632 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bfed0700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b8094e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.632 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bfed0700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 0x7fe3b8149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.632 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bfed0700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8095c40 con 0x7fe3b805b970 2022-11-27T15:01:48.633 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bfed0700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8095340 con 0x7fe3b8139a30 2022-11-27T15:01:48.633 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bfed0700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8094880 con 0x7fe3b80a2ea0 2022-11-27T15:01:48.633 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bdc6c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3b805b970 0x7fe3b8094200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.633 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bd46b700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b8094e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.633 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bdc6c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3b805b970 0x7fe3b8094200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51146/0 (socket says 172.21.15.39:51146) 2022-11-27T15:01:48.634 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bdc6c700 1 -- 172.21.15.39:0/450573147 learned_addr learned my addr 172.21.15.39:0/450573147 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:48.634 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3be46d700 1 --2- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 0x7fe3b8149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.634 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bdc6c700 1 -- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 msgr2=0x7fe3b8094e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:48.634 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bdc6c700 1 --2- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b8094e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.635 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bdc6c700 1 -- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 msgr2=0x7fe3b8149000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:48.635 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bdc6c700 1 --2- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 0x7fe3b8149000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.635 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bdc6c700 1 -- 172.21.15.39:0/450573147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b0009470 con 0x7fe3b805b970 2022-11-27T15:01:48.635 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3be46d700 1 --2- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 0x7fe3b8149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:48.636 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.619+0000 7fe3bd46b700 1 --2- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b8094e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:48.636 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.620+0000 7fe3bdc6c700 1 --2- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3b805b970 0x7fe3b8094200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe3a800c000 tx=0x7fe3a8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:48.636 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.620+0000 7fe3a6ffd700 1 -- 172.21.15.39:0/450573147 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3a8010780 con 0x7fe3b805b970 2022-11-27T15:01:48.636 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.620+0000 7fe3a6ffd700 1 -- 172.21.15.39:0/450573147 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a80108e0 con 0x7fe3b805b970 2022-11-27T15:01:48.636 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.620+0000 7fe3a6ffd700 1 -- 172.21.15.39:0/450573147 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3a8010c40 con 0x7fe3b805b970 2022-11-27T15:01:48.637 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.620+0000 7fe3bfed0700 1 -- 172.21.15.39:0/450573147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b400b1e0 con 0x7fe3b805b970 2022-11-27T15:01:48.637 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.620+0000 7fe3bfed0700 1 -- 172.21.15.39:0/450573147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b80960d0 con 0x7fe3b805b970 2022-11-27T15:01:48.637 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.621+0000 7fe3a6ffd700 1 -- 172.21.15.39:0/450573147 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fe3a801c460 con 0x7fe3b805b970 2022-11-27T15:01:48.637 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.621+0000 7fe3a6ffd700 1 -- 172.21.15.39:0/450573147 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fe3a804a8f0 con 0x7fe3b805b970 2022-11-27T15:01:48.638 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.621+0000 7fe3bfed0700 1 -- 172.21.15.39:0/450573147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b8096890 con 0x7fe3b805b970 2022-11-27T15:01:48.638 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.624+0000 7fe3a6ffd700 1 -- 172.21.15.39:0/450573147 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fe3a8033570 con 0x7fe3b805b970 2022-11-27T15:01:48.775 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.774+0000 7fe3bfed0700 1 -- 172.21.15.39:0/450573147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3b80a1d50 con 0x7fe3b805b970 2022-11-27T15:01:48.776 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.774+0000 7fe3a6ffd700 1 -- 172.21.15.39:0/450573147 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+1374 (secure 0 0 0) 0x7fe3a8033160 con 0x7fe3b805b970 2022-11-27T15:01:48.777 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:48.777 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4194,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6818/2768721463","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6818","nonce":2768721463}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6819/142076358","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6819","nonce":142076358}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.39:6817/927208116","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6817","nonce":927208116}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4}],"filesystems":[]} 2022-11-27T15:01:48.781 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.775+0000 7fe3bfed0700 1 -- 172.21.15.39:0/450573147 >> v1:172.21.15.39:6816/104537 conn(0x7fe398038da0 legacy=0x7fe39803b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:48.781 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.775+0000 7fe3bfed0700 1 -- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3b805b970 msgr2=0x7fe3b8094200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:48.781 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.775+0000 7fe3bfed0700 1 --2- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3b805b970 0x7fe3b8094200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe3a800c000 tx=0x7fe3a8006df0).stop 2022-11-27T15:01:48.782 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.776+0000 7fe3bfed0700 1 -- 172.21.15.39:0/450573147 shutdown_connections 2022-11-27T15:01:48.782 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.776+0000 7fe3bfed0700 1 --2- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3b805b970 0x7fe3b8094200 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.782 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.776+0000 7fe3bfed0700 1 --2- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3b8139a30 0x7fe3b8149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.782 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.776+0000 7fe3bfed0700 1 --2- 172.21.15.39:0/450573147 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe3b80a2ea0 0x7fe3b8094e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.776+0000 7fe3bfed0700 1 -- 172.21.15.39:0/450573147 >> 172.21.15.39:0/450573147 conn(0x7fe3b80a8a70 msgr2=0x7fe3b80a8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.776+0000 7fe3bfed0700 1 -- 172.21.15.39:0/450573147 shutdown_connections 2022-11-27T15:01:48.783 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.776+0000 7fe3bfed0700 1 -- 172.21.15.39:0/450573147 wait complete. 2022-11-27T15:01:48.784 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 4 2022-11-27T15:01:48.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T15:01:48.901 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.893+0000 7fbb75b6f700 1 Processor -- start 2022-11-27T15:01:48.902 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.893+0000 7fbb75b6f700 1 -- start start 2022-11-27T15:01:48.902 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb75b6f700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb70139a30 0x7fbb70139e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.902 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb75b6f700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb700a1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.902 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb75b6f700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbb7005b970 0x7fbb7005bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.903 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb75b6f700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb700a0b60 con 0x7fbb7005b970 2022-11-27T15:01:48.903 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb75b6f700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb701393b0 con 0x7fbb70139a30 2022-11-27T15:01:48.903 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb75b6f700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb701395e0 con 0x7fbb700a2ea0 2022-11-27T15:01:48.904 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6ffff700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbb7005b970 0x7fbb7005bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.904 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6effd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb700a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.904 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6f7fe700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb70139a30 0x7fbb70139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.904 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6ffff700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbb7005b970 0x7fbb7005bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51154/0 (socket says 172.21.15.39:51154) 2022-11-27T15:01:48.905 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6effd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb700a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:33984/0 (socket says 172.21.15.39:33984) 2022-11-27T15:01:48.905 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6ffff700 1 -- 172.21.15.39:0/2372805361 learned_addr learned my addr 172.21.15.39:0/2372805361 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:48.905 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6effd700 1 -- 172.21.15.39:0/2372805361 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb70139a30 msgr2=0x7fbb70139e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:48.905 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6effd700 1 --2- 172.21.15.39:0/2372805361 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb70139a30 0x7fbb70139e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.906 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6effd700 1 -- 172.21.15.39:0/2372805361 >> v2:172.21.15.39:3300/0 conn(0x7fbb7005b970 msgr2=0x7fbb7005bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:48.906 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6effd700 1 --2- 172.21.15.39:0/2372805361 >> v2:172.21.15.39:3300/0 conn(0x7fbb7005b970 0x7fbb7005bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.906 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6effd700 1 -- 172.21.15.39:0/2372805361 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700a1080 con 0x7fbb700a2ea0 2022-11-27T15:01:48.907 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6ffff700 1 --2- 172.21.15.39:0/2372805361 >> v2:172.21.15.39:3300/0 conn(0x7fbb7005b970 0x7fbb7005bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:48.907 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.894+0000 7fbb6effd700 1 --2- 172.21.15.39:0/2372805361 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb700a1860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fbb60002f70 tx=0x7fbb60006df0).ready entity=mon.2 client_cookie=dfc13d28902b1edc server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:48.908 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb6e7fc700 1 -- 172.21.15.39:0/2372805361 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb6000d4a0 con 0x7fbb700a2ea0 2022-11-27T15:01:48.908 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb6e7fc700 1 -- 172.21.15.39:0/2372805361 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60000e90 con 0x7fbb700a2ea0 2022-11-27T15:01:48.908 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb6e7fc700 1 -- 172.21.15.39:0/2372805361 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb6000d7f0 con 0x7fbb700a2ea0 2022-11-27T15:01:48.908 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb75b6f700 1 -- 172.21.15.39:0/2372805361 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 msgr2=0x7fbb700a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:48.909 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb75b6f700 1 --2- 172.21.15.39:0/2372805361 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb700a1860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fbb60002f70 tx=0x7fbb60006df0).stop 2022-11-27T15:01:48.909 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb75b6f700 1 -- 172.21.15.39:0/2372805361 shutdown_connections 2022-11-27T15:01:48.909 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb75b6f700 1 --2- 172.21.15.39:0/2372805361 >> v2:172.21.15.39:3300/0 conn(0x7fbb7005b970 0x7fbb7005bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.910 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb75b6f700 1 --2- 172.21.15.39:0/2372805361 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb70139a30 0x7fbb70139e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.910 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb75b6f700 1 --2- 172.21.15.39:0/2372805361 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb700a1860 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.910 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb75b6f700 1 -- 172.21.15.39:0/2372805361 >> 172.21.15.39:0/2372805361 conn(0x7fbb700a8a70 msgr2=0x7fbb700a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:48.910 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb75b6f700 1 -- 172.21.15.39:0/2372805361 shutdown_connections 2022-11-27T15:01:48.911 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.895+0000 7fbb75b6f700 1 -- 172.21.15.39:0/2372805361 wait complete. 2022-11-27T15:01:48.911 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.896+0000 7fbb75b6f700 1 Processor -- start 2022-11-27T15:01:48.911 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.896+0000 7fbb75b6f700 1 -- start start 2022-11-27T15:01:48.912 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb75b6f700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb7005b970 0x7fbb70149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.912 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb75b6f700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb70098900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.912 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb75b6f700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb70139a30 0x7fbb70097080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:48.912 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb75b6f700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70149fa0 con 0x7fbb70139a30 2022-11-27T15:01:48.913 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb75b6f700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70096dc0 con 0x7fbb7005b970 2022-11-27T15:01:48.913 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb75b6f700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70149560 con 0x7fbb700a2ea0 2022-11-27T15:01:48.913 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6effd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb70098900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.914 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6ffff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb70139a30 0x7fbb70097080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.914 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6f7fe700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb7005b970 0x7fbb70149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:48.914 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6effd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb70098900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:34000/0 (socket says 172.21.15.39:34000) 2022-11-27T15:01:48.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6effd700 1 -- 172.21.15.39:0/3754776177 learned_addr learned my addr 172.21.15.39:0/3754776177 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:48.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6f7fe700 1 -- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 msgr2=0x7fbb70098900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:48.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6f7fe700 1 --2- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb70098900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6f7fe700 1 -- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb70139a30 msgr2=0x7fbb70097080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:48.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6f7fe700 1 --2- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb70139a30 0x7fbb70097080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:48.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6f7fe700 1 -- 172.21.15.39:0/3754776177 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb68009530 con 0x7fbb7005b970 2022-11-27T15:01:48.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6effd700 1 --2- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb70098900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:48.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6f7fe700 1 --2- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb7005b970 0x7fbb70149ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbb5800c000 tx=0x7fbb58006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:48.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.897+0000 7fbb6cff9700 1 -- 172.21.15.39:0/3754776177 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb58011750 con 0x7fbb7005b970 2022-11-27T15:01:48.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.898+0000 7fbb6cff9700 1 -- 172.21.15.39:0/3754776177 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb580118b0 con 0x7fbb7005b970 2022-11-27T15:01:48.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.898+0000 7fbb6cff9700 1 -- 172.21.15.39:0/3754776177 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb58011c10 con 0x7fbb7005b970 2022-11-27T15:01:48.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.898+0000 7fbb75b6f700 1 -- 172.21.15.39:0/3754776177 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6000b180 con 0x7fbb7005b970 2022-11-27T15:01:48.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.898+0000 7fbb75b6f700 1 -- 172.21.15.39:0/3754776177 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb70095560 con 0x7fbb7005b970 2022-11-27T15:01:48.918 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.898+0000 7fbb6cff9700 1 -- 172.21.15.39:0/3754776177 <== mon.1 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fbb58011d70 con 0x7fbb7005b970 2022-11-27T15:01:48.918 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.899+0000 7fbb75b6f700 1 -- 172.21.15.39:0/3754776177 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb70149900 con 0x7fbb7005b970 2022-11-27T15:01:48.918 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.899+0000 7fbb6cff9700 1 -- 172.21.15.39:0/3754776177 <== mon.1 v2:172.21.15.39:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fbb5804a7d0 con 0x7fbb7005b970 2022-11-27T15:01:48.919 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:48.902+0000 7fbb6cff9700 1 -- 172.21.15.39:0/3754776177 <== mon.1 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fbb58033450 con 0x7fbb7005b970 2022-11-27T15:01:49.019 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.018+0000 7fbb75b6f700 1 -- 172.21.15.39:0/3754776177 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb700a1d50 con 0x7fbb7005b970 2022-11-27T15:01:49.020 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.019+0000 7fbb6cff9700 1 -- 172.21.15.39:0/3754776177 <== mon.1 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+7644 (secure 0 0 0) 0x7fbb58033040 con 0x7fbb7005b970 2022-11-27T15:01:49.021 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:49.021 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":15,"fsid":"fb73c991-5ff5-4983-821e-38ecbf33964f","created":"2022-11-27T15:01:32.796785+0000","modified":"2022-11-27T15:01:44.107526+0000","last_up_change":"2022-11-27T15:01:38.820289+0000","last_in_change":"2022-11-27T15:01:38.820289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-11-27T15:01:34.849418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T15:01:41.877873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"ba0fb473-e933-4ad1-a5f8-2cdd4138f3ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6800","nonce":105137}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6801","nonce":105137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6803","nonce":105137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6802","nonce":105137}]},"public_addr":"172.21.15.39:6800/105137","cluster_addr":"172.21.15.39:6801/105137","heartbeat_back_addr":"172.21.15.39:6803/105137","heartbeat_front_addr":"172.21.15.39:6802/105137","state":["exists","up"]},{"osd":1,"uuid":"d3e5e074-117b-4420-8340-e241cb3e6553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6805","nonce":105140}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6809","nonce":105140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6811","nonce":105140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6810","nonce":105140}]},"public_addr":"172.21.15.39:6805/105140","cluster_addr":"172.21.15.39:6809/105140","heartbeat_back_addr":"172.21.15.39:6811/105140","heartbeat_front_addr":"172.21.15.39:6810/105140","state":["exists","up"]},{"osd":2,"uuid":"2d655d83-8b81-4419-a44f-f4833633c621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6812","nonce":105139}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6813","nonce":105139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6815","nonce":105139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6814","nonce":105139}]},"public_addr":"172.21.15.39:6812/105139","cluster_addr":"172.21.15.39:6813/105139","heartbeat_back_addr":"172.21.15.39:6815/105139","heartbeat_front_addr":"172.21.15.39:6814/105139","state":["exists","up"]},{"osd":3,"uuid":"1f4c9a3a-a40b-43ae-8bd1-7672f6d9a51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6804","nonce":105138}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6806","nonce":105138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6808","nonce":105138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6807","nonce":105138}]},"public_addr":"172.21.15.39:6804/105138","cluster_addr":"172.21.15.39:6806/105138","heartbeat_back_addr":"172.21.15.39:6808/105138","heartbeat_front_addr":"172.21.15.39:6807/105138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.532356+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.825983+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.875364+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.883149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T15:01:49.024 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.020+0000 7fbb75b6f700 1 -- 172.21.15.39:0/3754776177 >> v1:172.21.15.39:6816/104537 conn(0x7fbb54038dd0 legacy=0x7fbb5403b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.024 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.020+0000 7fbb75b6f700 1 -- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb7005b970 msgr2=0x7fbb70149ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.025 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.020+0000 7fbb75b6f700 1 --2- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb7005b970 0x7fbb70149ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbb5800c000 tx=0x7fbb58006df0).stop 2022-11-27T15:01:49.025 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.020+0000 7fbb75b6f700 1 -- 172.21.15.39:0/3754776177 shutdown_connections 2022-11-27T15:01:49.025 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.020+0000 7fbb75b6f700 1 --2- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb70139a30 0x7fbb70097080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.026 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.020+0000 7fbb75b6f700 1 --2- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb7005b970 0x7fbb70149ab0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.026 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.020+0000 7fbb75b6f700 1 --2- 172.21.15.39:0/3754776177 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fbb700a2ea0 0x7fbb70098900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.027 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.020+0000 7fbb75b6f700 1 -- 172.21.15.39:0/3754776177 >> 172.21.15.39:0/3754776177 conn(0x7fbb700a8a70 msgr2=0x7fbb700a5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:49.027 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.020+0000 7fbb75b6f700 1 -- 172.21.15.39:0/3754776177 shutdown_connections 2022-11-27T15:01:49.028 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.020+0000 7fbb75b6f700 1 -- 172.21.15.39:0/3754776177 wait complete. 2022-11-27T15:01:49.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-11-27T15:01:34.849418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T15:01:41.877873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T15:01:49.034 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-11-27T15:01:49.163 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f718efdb700 1 Processor -- start 2022-11-27T15:01:49.164 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f718efdb700 1 -- start start 2022-11-27T15:01:49.164 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f718efdb700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7188139b70 0x7f718805be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f718efdb700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7188139f50 0x7f718805d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f718efdb700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7188139620 con 0x7f718805ccd0 2022-11-27T15:01:49.165 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f718efdb700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718805d110 con 0x7f7188139f50 2022-11-27T15:01:49.166 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f718efdb700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f718805dbb0 con 0x7f7188139b70 2022-11-27T15:01:49.166 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f718d578700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7188139f50 0x7f718805d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.167 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f718cd77700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7188139b70 0x7f718805be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.167 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f7187fff700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f718805ccd0 0x7f718805fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51048/0 (socket says 172.21.15.39:51048) 2022-11-27T15:01:49.167 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.143+0000 7f7187fff700 1 -- 172.21.15.39:0/1821229623 learned_addr learned my addr 172.21.15.39:0/1821229623 (peer_addr_for_me v1:172.21.15.39:0/0) 2022-11-27T15:01:49.168 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.144+0000 7f718d578700 1 -- 172.21.15.39:0/1821229623 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7188139b70 msgr2=0x7f718805be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:49.169 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.144+0000 7f718d578700 1 --2- 172.21.15.39:0/1821229623 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7188139b70 0x7f718805be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.170 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.144+0000 7f718d578700 1 -- 172.21.15.39:0/1821229623 >> v1:172.21.15.39:6789/0 conn(0x7f718805ccd0 legacy=0x7f718805fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.170 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.144+0000 7f718d578700 1 -- 172.21.15.39:0/1821229623 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718813a3e0 con 0x7f7188139f50 2022-11-27T15:01:49.170 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.144+0000 7f71877fe700 1 -- 172.21.15.39:0/1821229623 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320465508 0 0) 0x7f7188139620 con 0x7f718805ccd0 2022-11-27T15:01:49.171 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.144+0000 7f718cd77700 1 --2- 172.21.15.39:0/1821229623 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7188139b70 0x7f718805be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:49.171 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.144+0000 7f718d578700 1 --2- 172.21.15.39:0/1821229623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188139f50 0x7f718805d6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f71800032f0 tx=0x7f71800091d0).ready entity=mon.0 client_cookie=485b4d93a319804 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:49.171 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.144+0000 7f71877fe700 1 -- 172.21.15.39:0/1821229623 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f718000b6c0 con 0x7f7188139f50 2022-11-27T15:01:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.144+0000 7f71877fe700 1 -- 172.21.15.39:0/1821229623 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718000b820 con 0x7f7188139f50 2022-11-27T15:01:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.144+0000 7f71877fe700 1 -- 172.21.15.39:0/1821229623 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f718000b990 con 0x7f7188139f50 2022-11-27T15:01:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.145+0000 7f718efdb700 1 -- 172.21.15.39:0/1821229623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188139f50 msgr2=0x7f718805d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.172 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.145+0000 7f718efdb700 1 --2- 172.21.15.39:0/1821229623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188139f50 0x7f718805d6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f71800032f0 tx=0x7f71800091d0).stop 2022-11-27T15:01:49.173 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.145+0000 7f718efdb700 1 -- 172.21.15.39:0/1821229623 shutdown_connections 2022-11-27T15:01:49.173 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.145+0000 7f718efdb700 1 --2- 172.21.15.39:0/1821229623 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f7188139b70 0x7f718805be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.173 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.145+0000 7f718efdb700 1 --2- 172.21.15.39:0/1821229623 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188139f50 0x7f718805d6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.174 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.145+0000 7f718efdb700 1 -- 172.21.15.39:0/1821229623 >> 172.21.15.39:0/1821229623 conn(0x7f718806c640 msgr2=0x7f71880ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:49.174 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.145+0000 7f718efdb700 1 -- 172.21.15.39:0/1821229623 shutdown_connections 2022-11-27T15:01:49.175 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.145+0000 7f718efdb700 1 -- 172.21.15.39:0/1821229623 wait complete. 2022-11-27T15:01:49.175 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.145+0000 7f718efdb700 1 Processor -- start 2022-11-27T15:01:49.176 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.145+0000 7f718efdb700 1 -- start start 2022-11-27T15:01:49.176 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f718efdb700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f718805ccd0 0x7f71881188b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.177 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f718efdb700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188139b70 0x7f7188115e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.177 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f718efdb700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7188139f50 0x7f7188148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.178 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f718efdb700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7188117120 con 0x7f7188139b70 2022-11-27T15:01:49.178 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f718efdb700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71881173e0 con 0x7f7188139f50 2022-11-27T15:01:49.179 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f718efdb700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7188116310 con 0x7f718805ccd0 2022-11-27T15:01:49.180 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f718cd77700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f718805ccd0 0x7f71881188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.180 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f7187fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188139b70 0x7f7188115e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.180 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f718cd77700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f718805ccd0 0x7f71881188b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:34022/0 (socket says 172.21.15.39:34022) 2022-11-27T15:01:49.181 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f718cd77700 1 -- 172.21.15.39:0/2034542940 learned_addr learned my addr 172.21.15.39:0/2034542940 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:49.181 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.146+0000 7f718d578700 1 --2- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7188139f50 0x7f7188148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.181 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f7187fff700 1 -- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f718805ccd0 msgr2=0x7f71881188b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.181 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f7187fff700 1 --2- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f718805ccd0 0x7f71881188b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.182 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f7187fff700 1 -- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7188139f50 msgr2=0x7f7188148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.182 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f7187fff700 1 --2- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7188139f50 0x7f7188148a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.182 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f7187fff700 1 -- 172.21.15.39:0/2034542940 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c00b180 con 0x7f7188139b70 2022-11-27T15:01:49.182 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f718cd77700 1 --2- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f718805ccd0 0x7f71881188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:49.183 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f7187fff700 1 --2- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188139b70 0x7f7188115e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7174000bc0 tx=0x7f71740079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:49.183 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f7185ffb700 1 -- 172.21.15.39:0/2034542940 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f717400cdd0 con 0x7f7188139b70 2022-11-27T15:01:49.183 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f7185ffb700 1 -- 172.21.15.39:0/2034542940 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71740070d0 con 0x7f7188139b70 2022-11-27T15:01:49.183 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f718efdb700 1 -- 172.21.15.39:0/2034542940 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7180009470 con 0x7f7188139b70 2022-11-27T15:01:49.184 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f7185ffb700 1 -- 172.21.15.39:0/2034542940 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7174007430 con 0x7f7188139b70 2022-11-27T15:01:49.184 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.147+0000 7f718efdb700 1 -- 172.21.15.39:0/2034542940 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7188116d00 con 0x7f7188139b70 2022-11-27T15:01:49.184 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.148+0000 7f7185ffb700 1 -- 172.21.15.39:0/2034542940 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f7174010070 con 0x7f7188139b70 2022-11-27T15:01:49.185 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.148+0000 7f718efdb700 1 -- 172.21.15.39:0/2034542940 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718805c4f0 con 0x7f7188139b70 2022-11-27T15:01:49.185 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.148+0000 7f7185ffb700 1 -- 172.21.15.39:0/2034542940 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f71740499c0 con 0x7f7188139b70 2022-11-27T15:01:49.185 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.151+0000 7f7185ffb700 1 -- 172.21.15.39:0/2034542940 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f717402ec10 con 0x7f7188139b70 2022-11-27T15:01:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.268+0000 7f718efdb700 1 -- 172.21.15.39:0/2034542940 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7188057a60 con 0x7f7188139b70 2022-11-27T15:01:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.269+0000 7f7185ffb700 1 -- 172.21.15.39:0/2034542940 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v15) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7174018020 con 0x7f7188139b70 2022-11-27T15:01:49.271 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2022-11-27T15:01:49.273 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.270+0000 7f718efdb700 1 -- 172.21.15.39:0/2034542940 >> v1:172.21.15.39:6816/104537 conn(0x7f7168038e30 legacy=0x7f716803b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.274 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.270+0000 7f718efdb700 1 -- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188139b70 msgr2=0x7f7188115e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.274 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.270+0000 7f718efdb700 1 --2- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188139b70 0x7f7188115e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7174000bc0 tx=0x7f71740079e0).stop 2022-11-27T15:01:49.275 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.270+0000 7f718efdb700 1 -- 172.21.15.39:0/2034542940 shutdown_connections 2022-11-27T15:01:49.275 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.270+0000 7f718efdb700 1 --2- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7188139f50 0x7f7188148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.275 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.270+0000 7f718efdb700 1 --2- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f718805ccd0 0x7f71881188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.276 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.270+0000 7f718efdb700 1 --2- 172.21.15.39:0/2034542940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7188139b70 0x7f7188115e20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.276 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.270+0000 7f718efdb700 1 -- 172.21.15.39:0/2034542940 >> 172.21.15.39:0/2034542940 conn(0x7f718806c640 msgr2=0x7f718805fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:49.276 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.270+0000 7f718efdb700 1 -- 172.21.15.39:0/2034542940 shutdown_connections 2022-11-27T15:01:49.276 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.270+0000 7f718efdb700 1 -- 172.21.15.39:0/2034542940 wait complete. 2022-11-27T15:01:49.282 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T15:01:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.391+0000 7eff15130700 1 Processor -- start 2022-11-27T15:01:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.391+0000 7eff15130700 1 -- start start 2022-11-27T15:01:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.391+0000 7eff15130700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7eff10139b70 0x7eff1005be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.391+0000 7eff15130700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005ccd0 0x7eff10139f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.391+0000 7eff15130700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff10139620 con 0x7eff1005d6c0 2022-11-27T15:01:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.391+0000 7eff15130700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1013a440 con 0x7eff1005ccd0 2022-11-27T15:01:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.391+0000 7eff15130700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff1005d110 con 0x7eff10139b70 2022-11-27T15:01:49.400 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0f59e700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7eff1005d6c0 0x7eff1005fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51064/0 (socket says 172.21.15.39:51064) 2022-11-27T15:01:49.400 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0e59c700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005ccd0 0x7eff10139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.400 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0ed9d700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7eff10139b70 0x7eff1005be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.401 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0f59e700 1 -- 172.21.15.39:0/2696432913 learned_addr learned my addr 172.21.15.39:0/2696432913 (peer_addr_for_me v1:172.21.15.39:0/0) 2022-11-27T15:01:49.401 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0dd9b700 1 -- 172.21.15.39:0/2696432913 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881823829 0 0) 0x7eff10139620 con 0x7eff1005d6c0 2022-11-27T15:01:49.401 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0dd9b700 1 -- 172.21.15.39:0/2696432913 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efef4003260 con 0x7eff1005d6c0 2022-11-27T15:01:49.401 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0e59c700 1 -- 172.21.15.39:0/2696432913 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7eff10139b70 msgr2=0x7eff1005be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:49.401 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0e59c700 1 --2- 172.21.15.39:0/2696432913 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7eff10139b70 0x7eff1005be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.402 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0e59c700 1 -- 172.21.15.39:0/2696432913 >> v1:172.21.15.39:6789/0 conn(0x7eff1005d6c0 legacy=0x7eff1005fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.402 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0e59c700 1 -- 172.21.15.39:0/2696432913 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1005c4f0 con 0x7eff1005ccd0 2022-11-27T15:01:49.402 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.392+0000 7eff0e59c700 1 --2- 172.21.15.39:0/2696432913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005ccd0 0x7eff10139f50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7eff04002f70 tx=0x7eff04006df0).ready entity=mon.1 client_cookie=de1f8c17d9d23b45 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:49.403 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff0dd9b700 1 -- 172.21.15.39:0/2696432913 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff0400d570 con 0x7eff1005ccd0 2022-11-27T15:01:49.403 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff0dd9b700 1 -- 172.21.15.39:0/2696432913 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff04000e90 con 0x7eff1005ccd0 2022-11-27T15:01:49.403 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff0dd9b700 1 -- 172.21.15.39:0/2696432913 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff0400d8c0 con 0x7eff1005ccd0 2022-11-27T15:01:49.403 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff15130700 1 -- 172.21.15.39:0/2696432913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005ccd0 msgr2=0x7eff10139f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff15130700 1 --2- 172.21.15.39:0/2696432913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005ccd0 0x7eff10139f50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7eff04002f70 tx=0x7eff04006df0).stop 2022-11-27T15:01:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff15130700 1 -- 172.21.15.39:0/2696432913 shutdown_connections 2022-11-27T15:01:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff15130700 1 --2- 172.21.15.39:0/2696432913 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7eff10139b70 0x7eff1005be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff15130700 1 --2- 172.21.15.39:0/2696432913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005ccd0 0x7eff10139f50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff15130700 1 -- 172.21.15.39:0/2696432913 >> 172.21.15.39:0/2696432913 conn(0x7eff1006c640 msgr2=0x7eff100ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff15130700 1 -- 172.21.15.39:0/2696432913 shutdown_connections 2022-11-27T15:01:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.393+0000 7eff15130700 1 -- 172.21.15.39:0/2696432913 wait complete. 2022-11-27T15:01:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.394+0000 7eff15130700 1 Processor -- start 2022-11-27T15:01:49.406 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.394+0000 7eff15130700 1 -- start start 2022-11-27T15:01:49.406 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff15130700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1005ccd0 0x7eff10146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.406 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff15130700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005d6c0 0x7eff10148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.406 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff15130700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7eff10139b70 0x7eff101494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff15130700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff10146100 con 0x7eff1005ccd0 2022-11-27T15:01:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff15130700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff100a6630 con 0x7eff1005d6c0 2022-11-27T15:01:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff15130700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff10145f50 con 0x7eff10139b70 2022-11-27T15:01:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1005ccd0 0x7eff10146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.408 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0ed9d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1005ccd0 0x7eff10146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51184/0 (socket says 172.21.15.39:51184) 2022-11-27T15:01:49.408 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0ed9d700 1 -- 172.21.15.39:0/3714404991 learned_addr learned my addr 172.21.15.39:0/3714404991 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:49.408 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0f59e700 1 --2- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7eff10139b70 0x7eff101494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.408 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0e59c700 1 --2- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005d6c0 0x7eff10148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.409 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0e59c700 1 -- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7eff10139b70 msgr2=0x7eff101494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.409 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0e59c700 1 --2- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7eff10139b70 0x7eff101494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.409 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0e59c700 1 -- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1005ccd0 msgr2=0x7eff10146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.409 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0e59c700 1 --2- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1005ccd0 0x7eff10146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.409 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0e59c700 1 -- 172.21.15.39:0/3714404991 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0400b180 con 0x7eff1005d6c0 2022-11-27T15:01:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0ed9d700 1 --2- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1005ccd0 0x7eff10146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff0e59c700 1 --2- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005d6c0 0x7eff10148fc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7eff04002f70 tx=0x7eff04014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7efefffff700 1 -- 172.21.15.39:0/3714404991 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff04013de0 con 0x7eff1005d6c0 2022-11-27T15:01:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7efefffff700 1 -- 172.21.15.39:0/3714404991 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0401e400 con 0x7eff1005d6c0 2022-11-27T15:01:49.411 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff15130700 1 -- 172.21.15.39:0/3714404991 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff101185c0 con 0x7eff1005d6c0 2022-11-27T15:01:49.411 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7efefffff700 1 -- 172.21.15.39:0/3714404991 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff0401e760 con 0x7eff1005d6c0 2022-11-27T15:01:49.411 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.395+0000 7eff15130700 1 -- 172.21.15.39:0/3714404991 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff101148b0 con 0x7eff1005d6c0 2022-11-27T15:01:49.411 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.396+0000 7efefffff700 1 -- 172.21.15.39:0/3714404991 <== mon.1 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7eff04025030 con 0x7eff1005d6c0 2022-11-27T15:01:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.396+0000 7eff15130700 1 -- 172.21.15.39:0/3714404991 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff10116540 con 0x7eff1005d6c0 2022-11-27T15:01:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.396+0000 7efefffff700 1 -- 172.21.15.39:0/3714404991 <== mon.1 v2:172.21.15.39:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7eff0404b090 con 0x7eff1005d6c0 2022-11-27T15:01:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.400+0000 7efefffff700 1 -- 172.21.15.39:0/3714404991 <== mon.1 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7eff10116540 con 0x7eff1005d6c0 2022-11-27T15:01:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.517+0000 7eff15130700 1 -- 172.21.15.39:0/3714404991 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7eff10116540 con 0x7eff1005d6c0 2022-11-27T15:01:49.519 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.517+0000 7efefffff700 1 -- 172.21.15.39:0/3714404991 <== mon.1 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7eff10116540 con 0x7eff1005d6c0 2022-11-27T15:01:49.520 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 8 2022-11-27T15:01:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.518+0000 7eff15130700 1 -- 172.21.15.39:0/3714404991 >> v1:172.21.15.39:6816/104537 conn(0x7efeec038da0 legacy=0x7efeec03b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.518+0000 7eff15130700 1 -- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005d6c0 msgr2=0x7eff10148fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.518+0000 7eff15130700 1 --2- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005d6c0 0x7eff10148fc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7eff04002f70 tx=0x7eff04014000).stop 2022-11-27T15:01:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.518+0000 7eff15130700 1 -- 172.21.15.39:0/3714404991 shutdown_connections 2022-11-27T15:01:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.518+0000 7eff15130700 1 --2- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff1005ccd0 0x7eff10146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.518+0000 7eff15130700 1 --2- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7eff10139b70 0x7eff101494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.518+0000 7eff15130700 1 --2- 172.21.15.39:0/3714404991 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff1005d6c0 0x7eff10148fc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.518+0000 7eff15130700 1 -- 172.21.15.39:0/3714404991 >> 172.21.15.39:0/3714404991 conn(0x7eff1006c640 msgr2=0x7eff100abbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.518+0000 7eff15130700 1 -- 172.21.15.39:0/3714404991 shutdown_connections 2022-11-27T15:01:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.518+0000 7eff15130700 1 -- 172.21.15.39:0/3714404991 wait complete. 2022-11-27T15:01:49.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-11-27T15:01:49.651 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.638+0000 7fdcc8b46700 1 Processor -- start 2022-11-27T15:01:49.652 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.638+0000 7fdcc8b46700 1 -- start start 2022-11-27T15:01:49.653 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc8b46700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdcc40a3410 0x7fdcc4063b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.653 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc8b46700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.653 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc8b46700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc4064000 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.654 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc8b46700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc4062e90 con 0x7fdcc40a3410 2022-11-27T15:01:49.654 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc8b46700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40a4540 con 0x7fdcc40a5040 2022-11-27T15:01:49.655 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc1d9b700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.655 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc259c700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdcc40a3410 0x7fdcc4063b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.655 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc1d9b700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc40a37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:43648/0 (socket says 172.21.15.39:43648) 2022-11-27T15:01:49.656 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc1d9b700 1 -- 172.21.15.39:0/3404818633 learned_addr learned my addr 172.21.15.39:0/3404818633 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:49.656 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc259c700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdcc40a3410 0x7fdcc4063b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51190/0 (socket says 172.21.15.39:51190) 2022-11-27T15:01:49.657 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc1d9b700 1 -- 172.21.15.39:0/3404818633 >> v2:172.21.15.39:3300/0 conn(0x7fdcc40a3410 msgr2=0x7fdcc4063b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:49.657 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc1d9b700 1 --2- 172.21.15.39:0/3404818633 >> v2:172.21.15.39:3300/0 conn(0x7fdcc40a3410 0x7fdcc4063b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.658 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc1d9b700 1 -- 172.21.15.39:0/3404818633 >> v1:172.21.15.39:6789/0 conn(0x7fdcc40a3ce0 legacy=0x7fdcc41362a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.658 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc1d9b700 1 -- 172.21.15.39:0/3404818633 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc40a2c90 con 0x7fdcc40a5040 2022-11-27T15:01:49.659 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.639+0000 7fdcc159a700 1 -- 172.21.15.39:0/3404818633 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512899958 0 0) 0x7fdcbc001490 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.659 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc259c700 1 --2- 172.21.15.39:0/3404818633 >> v2:172.21.15.39:3300/0 conn(0x7fdcc40a3410 0x7fdcc4063b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:49.660 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc1d9b700 1 --2- 172.21.15.39:0/3404818633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc40a37f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fdcc40a47e0 tx=0x7fdcb4006eb0).ready entity=mon.1 client_cookie=aeea41ae6f56088d server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:49.660 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc159a700 1 -- 172.21.15.39:0/3404818633 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcb400d5c0 con 0x7fdcc40a5040 2022-11-27T15:01:49.660 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc159a700 1 -- 172.21.15.39:0/3404818633 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4000e90 con 0x7fdcc40a5040 2022-11-27T15:01:49.661 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc159a700 1 -- 172.21.15.39:0/3404818633 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcb400d910 con 0x7fdcc40a5040 2022-11-27T15:01:49.661 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc8b46700 1 -- 172.21.15.39:0/3404818633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 msgr2=0x7fdcc40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.662 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc8b46700 1 --2- 172.21.15.39:0/3404818633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc40a37f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fdcc40a47e0 tx=0x7fdcb4006eb0).stop 2022-11-27T15:01:49.662 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc8b46700 1 -- 172.21.15.39:0/3404818633 shutdown_connections 2022-11-27T15:01:49.662 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc8b46700 1 --2- 172.21.15.39:0/3404818633 >> v2:172.21.15.39:3300/0 conn(0x7fdcc40a3410 0x7fdcc4063b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.663 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc8b46700 1 --2- 172.21.15.39:0/3404818633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc40a37f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.663 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc8b46700 1 -- 172.21.15.39:0/3404818633 >> 172.21.15.39:0/3404818633 conn(0x7fdcc40a8a70 msgr2=0x7fdcc40a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:49.664 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc8b46700 1 -- 172.21.15.39:0/3404818633 shutdown_connections 2022-11-27T15:01:49.664 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.640+0000 7fdcc8b46700 1 -- 172.21.15.39:0/3404818633 wait complete. 2022-11-27T15:01:49.664 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.641+0000 7fdcc8b46700 1 Processor -- start 2022-11-27T15:01:49.665 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.641+0000 7fdcc8b46700 1 -- start start 2022-11-27T15:01:49.665 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc8b46700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fdcc40a3410 0x7fdcc40926e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.666 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc8b46700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc40a3ce0 0x7fdcc409e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.666 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc8b46700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc409ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.667 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc8b46700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40a09d0 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.667 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc8b46700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40a0c90 con 0x7fdcc40a5040 2022-11-27T15:01:49.668 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc8b46700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40a24a0 con 0x7fdcc40a3410 2022-11-27T15:01:49.668 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc1d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc40a3ce0 0x7fdcc409e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.668 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc259c700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fdcc40a3410 0x7fdcc40926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.669 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc1d9b700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc40a3ce0 0x7fdcc409e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51200/0 (socket says 172.21.15.39:51200) 2022-11-27T15:01:49.669 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc2d9d700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc409ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.669 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc1d9b700 1 -- 172.21.15.39:0/2802210386 learned_addr learned my addr 172.21.15.39:0/2802210386 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:49.670 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc259c700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fdcc40a3410 0x7fdcc40926e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:34054/0 (socket says 172.21.15.39:34054) 2022-11-27T15:01:49.670 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc1d9b700 1 -- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fdcc40a3410 msgr2=0x7fdcc40926e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.670 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc1d9b700 1 --2- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fdcc40a3410 0x7fdcc40926e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.671 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc1d9b700 1 -- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 msgr2=0x7fdcc409ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.671 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc1d9b700 1 --2- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc409ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.672 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc1d9b700 1 -- 172.21.15.39:0/2802210386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb800b180 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.672 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc2d9d700 1 --2- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc409ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:49.673 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.642+0000 7fdcc1d9b700 1 --2- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc40a3ce0 0x7fdcc409e7f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fdcc40a47e0 tx=0x7fdcb4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:49.673 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.643+0000 7fdcb37fe700 1 -- 172.21.15.39:0/2802210386 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcb4002f70 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.674 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.643+0000 7fdcb37fe700 1 -- 172.21.15.39:0/2802210386 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb400de40 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.674 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.643+0000 7fdcb37fe700 1 -- 172.21.15.39:0/2802210386 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcb4011d70 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.675 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.643+0000 7fdcc8b46700 1 -- 172.21.15.39:0/2802210386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb400b1e0 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.675 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.643+0000 7fdcc8b46700 1 -- 172.21.15.39:0/2802210386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc40a1260 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.675 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.644+0000 7fdcb37fe700 1 -- 172.21.15.39:0/2802210386 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fdcb4024030 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.676 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.644+0000 7fdcc8b46700 1 -- 172.21.15.39:0/2802210386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc40a0770 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.676 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.644+0000 7fdcb37fe700 1 -- 172.21.15.39:0/2802210386 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fdcb404b450 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.677 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.647+0000 7fdcb37fe700 1 -- 172.21.15.39:0/2802210386 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fdcb401f200 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.766 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.764+0000 7fdcc8b46700 1 -- 172.21.15.39:0/2802210386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdcc40a2f30 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.766 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.765+0000 7fdcb37fe700 1 -- 172.21.15.39:0/2802210386 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v15) v1 ==== 84+0+143 (secure 0 0 0) 0x7fdcb4027440 con 0x7fdcc40a3ce0 2022-11-27T15:01:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:49.767 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2022-11-27T15:01:49.767 INFO:teuthology.orchestra.run.smithi039.stdout: { 2022-11-27T15:01:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2022-11-27T15:01:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "device_health_metrics" 2022-11-27T15:01:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2022-11-27T15:01:49.768 INFO:teuthology.orchestra.run.smithi039.stdout: { 2022-11-27T15:01:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 2, 2022-11-27T15:01:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "rbd" 2022-11-27T15:01:49.769 INFO:teuthology.orchestra.run.smithi039.stdout: } 2022-11-27T15:01:49.769 INFO:teuthology.orchestra.run.smithi039.stdout:] 2022-11-27T15:01:49.770 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.766+0000 7fdcc8b46700 1 -- 172.21.15.39:0/2802210386 >> v1:172.21.15.39:6816/104537 conn(0x7fdca0038dd0 legacy=0x7fdca003b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.766+0000 7fdcc8b46700 1 -- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc40a3ce0 msgr2=0x7fdcc409e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.766+0000 7fdcc8b46700 1 --2- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc40a3ce0 0x7fdcc409e7f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fdcc40a47e0 tx=0x7fdcb4015000).stop 2022-11-27T15:01:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.766+0000 7fdcc8b46700 1 -- 172.21.15.39:0/2802210386 shutdown_connections 2022-11-27T15:01:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.766+0000 7fdcc8b46700 1 --2- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcc40a5040 0x7fdcc409ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.766+0000 7fdcc8b46700 1 --2- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fdcc40a3410 0x7fdcc40926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.766+0000 7fdcc8b46700 1 --2- 172.21.15.39:0/2802210386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc40a3ce0 0x7fdcc409e7f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.766+0000 7fdcc8b46700 1 -- 172.21.15.39:0/2802210386 >> 172.21.15.39:0/2802210386 conn(0x7fdcc40a8a70 msgr2=0x7fdcc405c800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.766+0000 7fdcc8b46700 1 -- 172.21.15.39:0/2802210386 shutdown_connections 2022-11-27T15:01:49.773 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.766+0000 7fdcc8b46700 1 -- 172.21.15.39:0/2802210386 wait complete. 2022-11-27T15:01:49.778 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-11-27T15:01:49.778 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2022-11-27T15:01:49.900 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.887+0000 7f85be99d700 1 Processor -- start 2022-11-27T15:01:49.901 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.887+0000 7f85be99d700 1 -- start start 2022-11-27T15:01:49.901 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.887+0000 7f85be99d700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 0x7f85b8133730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.902 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.887+0000 7f85be99d700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805d5d0 0x7f85b805cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.902 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85be99d700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85b805fd70 0x7f85b80601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.902 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85be99d700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b8139580 con 0x7f85b805fd70 2022-11-27T15:01:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85be99d700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b805c250 con 0x7f85b805d5d0 2022-11-27T15:01:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85be99d700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81397b0 con 0x7f85b8133350 2022-11-27T15:01:49.904 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805d5d0 0x7f85b805cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.904 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 0x7f85b8133730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.904 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85bcf3a700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85b805fd70 0x7f85b80601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.905 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805d5d0 0x7f85b805cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:43660/0 (socket says 172.21.15.39:43660) 2022-11-27T15:01:49.905 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85b77fe700 1 -- 172.21.15.39:0/1294651252 learned_addr learned my addr 172.21.15.39:0/1294651252 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:49.906 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85bcf3a700 1 -- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 msgr2=0x7f85b8133730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:49.906 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85bcf3a700 1 --2- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 0x7f85b8133730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.906 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85bcf3a700 1 -- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805d5d0 msgr2=0x7f85b805cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:49.907 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85bcf3a700 1 --2- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805d5d0 0x7f85b805cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.907 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85bcf3a700 1 -- 172.21.15.39:0/1294651252 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b8139a50 con 0x7f85b805fd70 2022-11-27T15:01:49.907 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85b77fe700 1 --2- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805d5d0 0x7f85b805cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:49.908 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85bcf3a700 1 --2- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805fd70 0x7f85b80601f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f85b00032f0 tx=0x7f85b00091d0).ready entity=mon.0 client_cookie=2063c8586c5d95f5 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:49.908 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85b6ffd700 1 -- 172.21.15.39:0/1294651252 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85b000b600 con 0x7f85b805fd70 2022-11-27T15:01:49.908 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85b6ffd700 1 -- 172.21.15.39:0/1294651252 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b000b760 con 0x7f85b805fd70 2022-11-27T15:01:49.908 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.888+0000 7f85b6ffd700 1 -- 172.21.15.39:0/1294651252 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85b000b8d0 con 0x7f85b805fd70 2022-11-27T15:01:49.909 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.889+0000 7f85be99d700 1 -- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805fd70 msgr2=0x7f85b80601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.909 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.889+0000 7f85be99d700 1 --2- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805fd70 0x7f85b80601f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f85b00032f0 tx=0x7f85b00091d0).stop 2022-11-27T15:01:49.909 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.889+0000 7f85be99d700 1 -- 172.21.15.39:0/1294651252 shutdown_connections 2022-11-27T15:01:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.889+0000 7f85be99d700 1 --2- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805fd70 0x7f85b80601f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.889+0000 7f85be99d700 1 --2- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 0x7f85b8133730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.889+0000 7f85be99d700 1 --2- 172.21.15.39:0/1294651252 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805d5d0 0x7f85b805cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.889+0000 7f85be99d700 1 -- 172.21.15.39:0/1294651252 >> 172.21.15.39:0/1294651252 conn(0x7f85b8066e50 msgr2=0x7f85b80a8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:49.911 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.889+0000 7f85be99d700 1 -- 172.21.15.39:0/1294651252 shutdown_connections 2022-11-27T15:01:49.911 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.889+0000 7f85be99d700 1 -- 172.21.15.39:0/1294651252 wait complete. 2022-11-27T15:01:49.911 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.890+0000 7f85be99d700 1 Processor -- start 2022-11-27T15:01:49.911 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.890+0000 7f85be99d700 1 -- start start 2022-11-27T15:01:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.890+0000 7f85be99d700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805d5d0 0x7f85b80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.890+0000 7f85be99d700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805fd70 0x7f85b8148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.890+0000 7f85be99d700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 0x7f85b81491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:49.913 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85be99d700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80a51a0 con 0x7f85b805d5d0 2022-11-27T15:01:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85be99d700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80a58d0 con 0x7f85b805fd70 2022-11-27T15:01:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85be99d700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80a5ad0 con 0x7f85b8133350 2022-11-27T15:01:49.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85bcf3a700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 0x7f85b81491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805d5d0 0x7f85b80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.915 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85bcf3a700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 0x7f85b81491b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:34062/0 (socket says 172.21.15.39:34062) 2022-11-27T15:01:49.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85bcf3a700 1 -- 172.21.15.39:0/2428475857 learned_addr learned my addr 172.21.15.39:0/2428475857 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:49.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85b7fff700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805d5d0 0x7f85b80a3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51218/0 (socket says 172.21.15.39:51218) 2022-11-27T15:01:49.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85b77fe700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805fd70 0x7f85b8148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:49.916 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85bcf3a700 1 -- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805fd70 msgr2=0x7f85b8148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85bcf3a700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805fd70 0x7f85b8148cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85bcf3a700 1 -- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805d5d0 msgr2=0x7f85b80a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85bcf3a700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805d5d0 0x7f85b80a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85bcf3a700 1 -- 172.21.15.39:0/2428475857 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a800b1e0 con 0x7f85b8133350 2022-11-27T15:01:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85b77fe700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805fd70 0x7f85b8148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-11-27T15:01:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85b7fff700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805d5d0 0x7f85b80a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85bcf3a700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 0x7f85b81491b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f85b00032f0 tx=0x7f85b0016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85b57fa700 1 -- 172.21.15.39:0/2428475857 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85b000b600 con 0x7f85b8133350 2022-11-27T15:01:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85b57fa700 1 -- 172.21.15.39:0/2428475857 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b0011c00 con 0x7f85b8133350 2022-11-27T15:01:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85b57fa700 1 -- 172.21.15.39:0/2428475857 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85b0006b00 con 0x7f85b8133350 2022-11-27T15:01:49.919 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85be99d700 1 -- 172.21.15.39:0/2428475857 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b0009470 con 0x7f85b8133350 2022-11-27T15:01:49.919 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.891+0000 7f85be99d700 1 -- 172.21.15.39:0/2428475857 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b80a43f0 con 0x7f85b8133350 2022-11-27T15:01:49.919 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.892+0000 7f85b57fa700 1 -- 172.21.15.39:0/2428475857 <== mon.2 v2:172.21.15.39:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f85b0026030 con 0x7f85b8133350 2022-11-27T15:01:49.919 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.892+0000 7f85be99d700 1 -- 172.21.15.39:0/2428475857 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b805d060 con 0x7f85b8133350 2022-11-27T15:01:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.892+0000 7f85b57fa700 1 -- 172.21.15.39:0/2428475857 <== mon.2 v2:172.21.15.39:3302/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f85b004afe0 con 0x7f85b8133350 2022-11-27T15:01:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:49.896+0000 7f85b57fa700 1 -- 172.21.15.39:0/2428475857 <== mon.2 v2:172.21.15.39:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f85b0002f40 con 0x7f85b8133350 2022-11-27T15:01:50.015 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.014+0000 7f85be99d700 1 -- 172.21.15.39:0/2428475857 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f85b80a52e0 con 0x7f85b8133350 2022-11-27T15:01:50.573 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.570+0000 7f85b57fa700 1 -- 172.21.15.39:0/2428475857 <== mon.2 v2:172.21.15.39:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v16) v1 ==== 138+0+0 (secure 0 0 0) 0x7f85b0024070 con 0x7f85b8133350 2022-11-27T15:01:50.580 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.578+0000 7f85be99d700 1 -- 172.21.15.39:0/2428475857 >> v1:172.21.15.39:6816/104537 conn(0x7f859c038da0 legacy=0x7f859c03b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:50.580 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.578+0000 7f85be99d700 1 -- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 msgr2=0x7f85b81491b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:50.580 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.578+0000 7f85be99d700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 0x7f85b81491b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f85b00032f0 tx=0x7f85b0016000).stop 2022-11-27T15:01:50.581 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.578+0000 7f85be99d700 1 -- 172.21.15.39:0/2428475857 shutdown_connections 2022-11-27T15:01:50.581 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.578+0000 7f85be99d700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b805d5d0 0x7f85b80a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.578+0000 7f85be99d700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f85b8133350 0x7f85b81491b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.578+0000 7f85be99d700 1 --2- 172.21.15.39:0/2428475857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85b805fd70 0x7f85b8148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.578+0000 7f85be99d700 1 -- 172.21.15.39:0/2428475857 >> 172.21.15.39:0/2428475857 conn(0x7f85b8066e50 msgr2=0x7f85b8131c30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.578+0000 7f85be99d700 1 -- 172.21.15.39:0/2428475857 shutdown_connections 2022-11-27T15:01:50.582 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.578+0000 7f85be99d700 1 -- 172.21.15.39:0/2428475857 wait complete. 2022-11-27T15:01:50.583 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2022-11-27T15:01:50.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2022-11-27T15:01:50.718 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.707+0000 7f1899941700 1 Processor -- start 2022-11-27T15:01:50.718 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.707+0000 7f1899941700 1 -- start start 2022-11-27T15:01:50.719 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1899941700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18940a9830 0x7f18941332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:50.719 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1899941700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405ccd0 0x7f189405d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:50.720 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1899941700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18940a9c10 con 0x7f189405fd70 2022-11-27T15:01:50.720 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1899941700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1894139450 con 0x7f18940a9830 2022-11-27T15:01:50.720 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1899941700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18941396f0 con 0x7f189405ccd0 2022-11-27T15:01:50.721 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1892ffd700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18940a9830 0x7f18941332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:50.722 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f18927fc700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405ccd0 0x7f189405d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:50.722 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f18937fe700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f189405fd70 0x7f18940601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51086/0 (socket says 172.21.15.39:51086) 2022-11-27T15:01:50.723 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f18937fe700 1 -- 172.21.15.39:0/2150691546 learned_addr learned my addr 172.21.15.39:0/2150691546 (peer_addr_for_me v1:172.21.15.39:0/0) 2022-11-27T15:01:50.723 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1892ffd700 1 -- 172.21.15.39:0/2150691546 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405ccd0 msgr2=0x7f189405d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:50.723 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1892ffd700 1 --2- 172.21.15.39:0/2150691546 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405ccd0 0x7f189405d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:50.723 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1892ffd700 1 -- 172.21.15.39:0/2150691546 >> v1:172.21.15.39:6789/0 conn(0x7f189405fd70 legacy=0x7f18940601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:50.723 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1891ffb700 1 -- 172.21.15.39:0/2150691546 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527384559 0 0) 0x7f18940a9c10 con 0x7f189405fd70 2022-11-27T15:01:50.724 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1892ffd700 1 -- 172.21.15.39:0/2150691546 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940a81b0 con 0x7f18940a9830 2022-11-27T15:01:50.724 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f18927fc700 1 --2- 172.21.15.39:0/2150691546 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405ccd0 0x7f189405d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:50.724 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1892ffd700 1 --2- 172.21.15.39:0/2150691546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 0x7f18941332c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1884002f70 tx=0x7f1884006df0).ready entity=mon.0 client_cookie=1208ead746c07cf2 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:50.724 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.708+0000 7f1891ffb700 1 -- 172.21.15.39:0/2150691546 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188400d570 con 0x7f18940a9830 2022-11-27T15:01:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.709+0000 7f1891ffb700 1 -- 172.21.15.39:0/2150691546 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1884000e90 con 0x7f18940a9830 2022-11-27T15:01:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.709+0000 7f1891ffb700 1 -- 172.21.15.39:0/2150691546 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188400d8c0 con 0x7f18940a9830 2022-11-27T15:01:50.726 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.709+0000 7f1899941700 1 -- 172.21.15.39:0/2150691546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 msgr2=0x7f18941332c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:50.726 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.709+0000 7f1899941700 1 --2- 172.21.15.39:0/2150691546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 0x7f18941332c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1884002f70 tx=0x7f1884006df0).stop 2022-11-27T15:01:50.726 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.709+0000 7f1899941700 1 -- 172.21.15.39:0/2150691546 shutdown_connections 2022-11-27T15:01:50.727 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.709+0000 7f1899941700 1 --2- 172.21.15.39:0/2150691546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 0x7f18941332c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:50.727 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.709+0000 7f1899941700 1 --2- 172.21.15.39:0/2150691546 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405ccd0 0x7f189405d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:50.728 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.709+0000 7f1899941700 1 -- 172.21.15.39:0/2150691546 >> 172.21.15.39:0/2150691546 conn(0x7f1894066d30 msgr2=0x7f18940a8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:50.728 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.709+0000 7f1899941700 1 -- 172.21.15.39:0/2150691546 shutdown_connections 2022-11-27T15:01:50.729 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.709+0000 7f1899941700 1 -- 172.21.15.39:0/2150691546 wait complete. 2022-11-27T15:01:50.729 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.710+0000 7f1899941700 1 Processor -- start 2022-11-27T15:01:50.729 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.710+0000 7f1899941700 1 -- start start 2022-11-27T15:01:50.730 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1899941700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189405ccd0 0x7f18940a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:50.730 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1899941700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405fd70 0x7f189412f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:50.731 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1899941700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 0x7f189412fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:50.731 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1899941700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18940a46f0 con 0x7f18940a9830 2022-11-27T15:01:50.732 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1899941700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18940a5ea0 con 0x7f189405ccd0 2022-11-27T15:01:50.732 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1899941700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18940a3880 con 0x7f189405fd70 2022-11-27T15:01:50.733 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1892ffd700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189405ccd0 0x7f18940a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:50.733 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18937fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 0x7f189412fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:50.733 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18927fc700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405fd70 0x7f189412f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:50.733 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1892ffd700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189405ccd0 0x7f18940a3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:43670/0 (socket says 172.21.15.39:43670) 2022-11-27T15:01:50.734 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18927fc700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405fd70 0x7f189412f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:34074/0 (socket says 172.21.15.39:34074) 2022-11-27T15:01:50.734 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18937fe700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 0x7f189412fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51234/0 (socket says 172.21.15.39:51234) 2022-11-27T15:01:50.734 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1892ffd700 1 -- 172.21.15.39:0/3776901045 learned_addr learned my addr 172.21.15.39:0/3776901045 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:50.734 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18937fe700 1 -- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405fd70 msgr2=0x7f189412f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:50.735 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18937fe700 1 --2- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405fd70 0x7f189412f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:50.735 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18937fe700 1 -- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189405ccd0 msgr2=0x7f18940a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:50.735 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18937fe700 1 --2- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189405ccd0 0x7f18940a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:50.735 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18937fe700 1 -- 172.21.15.39:0/3776901045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188800b180 con 0x7f18940a9830 2022-11-27T15:01:50.736 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18927fc700 1 --2- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405fd70 0x7f189412f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:50.736 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1892ffd700 1 --2- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189405ccd0 0x7f18940a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:50.736 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f18937fe700 1 --2- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 0x7f189412fad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f188c001250 tx=0x7f188c007d00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:50.736 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1883fff700 1 -- 172.21.15.39:0/3776901045 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188c005ba0 con 0x7f18940a9830 2022-11-27T15:01:50.736 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1883fff700 1 -- 172.21.15.39:0/3776901045 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c005d00 con 0x7f18940a9830 2022-11-27T15:01:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1899941700 1 -- 172.21.15.39:0/3776901045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188400b180 con 0x7f18940a9830 2022-11-27T15:01:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1883fff700 1 -- 172.21.15.39:0/3776901045 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188c004a90 con 0x7f18940a9830 2022-11-27T15:01:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.711+0000 7f1899941700 1 -- 172.21.15.39:0/3776901045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18940a5c60 con 0x7f18940a9830 2022-11-27T15:01:50.737 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.713+0000 7f1883fff700 1 -- 172.21.15.39:0/3776901045 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f188c004bf0 con 0x7f18940a9830 2022-11-27T15:01:50.738 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.713+0000 7f1899941700 1 -- 172.21.15.39:0/3776901045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18940a6010 con 0x7f18940a9830 2022-11-27T15:01:50.738 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.713+0000 7f1883fff700 1 -- 172.21.15.39:0/3776901045 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3192+0+0 (secure 0 0 0) 0x7f188c04a120 con 0x7f18940a9830 2022-11-27T15:01:50.738 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.716+0000 7f1883fff700 1 -- 172.21.15.39:0/3776901045 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f188c02f200 con 0x7f18940a9830 2022-11-27T15:01:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:50.832+0000 7f1899941700 1 -- 172.21.15.39:0/3776901045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f18941337b0 con 0x7f18940a9830 2022-11-27T15:01:51.573 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.572+0000 7f1883fff700 1 -- 172.21.15.39:0/3776901045 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v17) v1 ==== 170+0+0 (secure 0 0 0) 0x7f188c032e60 con 0x7f18940a9830 2022-11-27T15:01:51.600 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.598+0000 7f1899941700 1 -- 172.21.15.39:0/3776901045 >> v1:172.21.15.39:6816/104537 conn(0x7f1870038e60 legacy=0x7f187003b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:51.601 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.598+0000 7f1899941700 1 -- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 msgr2=0x7f189412fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:51.601 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.598+0000 7f1899941700 1 --2- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 0x7f189412fad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f188c001250 tx=0x7f188c007d00).stop 2022-11-27T15:01:51.601 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.599+0000 7f1899941700 1 -- 172.21.15.39:0/3776901045 shutdown_connections 2022-11-27T15:01:51.602 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.599+0000 7f1899941700 1 --2- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940a9830 0x7f189412fad0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:51.602 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.599+0000 7f1899941700 1 --2- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189405ccd0 0x7f18940a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:51.602 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.599+0000 7f1899941700 1 --2- 172.21.15.39:0/3776901045 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f189405fd70 0x7f189412f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:51.603 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.599+0000 7f1899941700 1 -- 172.21.15.39:0/3776901045 >> 172.21.15.39:0/3776901045 conn(0x7f1894066d30 msgr2=0x7f1894135e50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:51.603 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.600+0000 7f1899941700 1 -- 172.21.15.39:0/3776901045 shutdown_connections 2022-11-27T15:01:51.604 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.601+0000 7f1899941700 1 -- 172.21.15.39:0/3776901045 wait complete. 2022-11-27T15:01:51.606 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2022-11-27T15:01:51.627 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-11-27T15:01:51.749 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.737+0000 7fc49f4d6700 1 Processor -- start 2022-11-27T15:01:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.738+0000 7fc49f4d6700 1 -- start start 2022-11-27T15:01:51.750 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.738+0000 7fc49f4d6700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc498139b70 0x7fc49805be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.738+0000 7fc49f4d6700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc49805ccd0 0x7fc498139f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:51.751 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.738+0000 7fc49f4d6700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc498139620 con 0x7fc49805d6c0 2022-11-27T15:01:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.738+0000 7fc49f4d6700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49813a440 con 0x7fc49805ccd0 2022-11-27T15:01:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.738+0000 7fc49f4d6700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49805d110 con 0x7fc498139b70 2022-11-27T15:01:51.752 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.738+0000 7fc49d272700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc498139b70 0x7fc49805be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:51.753 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.738+0000 7fc49d272700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc498139b70 0x7fc49805be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:43678/0 (socket says 172.21.15.39:43678) 2022-11-27T15:01:51.753 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.738+0000 7fc49d272700 1 -- 172.21.15.39:0/2616062971 learned_addr learned my addr 172.21.15.39:0/2616062971 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:51.753 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.738+0000 7fc49ca71700 1 --2- 172.21.15.39:0/2616062971 >> v2:172.21.15.39:3300/0 conn(0x7fc49805ccd0 0x7fc498139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:51.753 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49d272700 1 -- 172.21.15.39:0/2616062971 >> v2:172.21.15.39:3300/0 conn(0x7fc49805ccd0 msgr2=0x7fc498139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:51.754 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49d272700 1 --2- 172.21.15.39:0/2616062971 >> v2:172.21.15.39:3300/0 conn(0x7fc49805ccd0 0x7fc498139f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:51.754 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49d272700 1 -- 172.21.15.39:0/2616062971 >> v1:172.21.15.39:6789/0 conn(0x7fc49805d6c0 legacy=0x7fc49805fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:51.754 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49d272700 1 -- 172.21.15.39:0/2616062971 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49805c4f0 con 0x7fc498139b70 2022-11-27T15:01:51.754 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49ca71700 1 --2- 172.21.15.39:0/2616062971 >> v2:172.21.15.39:3300/0 conn(0x7fc49805ccd0 0x7fc498139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:51.755 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49d272700 1 --2- 172.21.15.39:0/2616062971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc498139b70 0x7fc49805be80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc494002f70 tx=0x7fc494006eb0).ready entity=mon.1 client_cookie=509a57b94782a166 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:51.755 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc48bfff700 1 -- 172.21.15.39:0/2616062971 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc49400d570 con 0x7fc498139b70 2022-11-27T15:01:51.755 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc48bfff700 1 -- 172.21.15.39:0/2616062971 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc494000e90 con 0x7fc498139b70 2022-11-27T15:01:51.755 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc48bfff700 1 -- 172.21.15.39:0/2616062971 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc49400d900 con 0x7fc498139b70 2022-11-27T15:01:51.756 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49f4d6700 1 -- 172.21.15.39:0/2616062971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc498139b70 msgr2=0x7fc49805be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:51.756 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49f4d6700 1 --2- 172.21.15.39:0/2616062971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc498139b70 0x7fc49805be80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc494002f70 tx=0x7fc494006eb0).stop 2022-11-27T15:01:51.756 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49f4d6700 1 -- 172.21.15.39:0/2616062971 shutdown_connections 2022-11-27T15:01:51.756 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49f4d6700 1 --2- 172.21.15.39:0/2616062971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc498139b70 0x7fc49805be80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:51.757 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49f4d6700 1 --2- 172.21.15.39:0/2616062971 >> v2:172.21.15.39:3300/0 conn(0x7fc49805ccd0 0x7fc498139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:51.757 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.739+0000 7fc49f4d6700 1 -- 172.21.15.39:0/2616062971 >> 172.21.15.39:0/2616062971 conn(0x7fc49806c640 msgr2=0x7fc4980ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:51.757 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.740+0000 7fc49f4d6700 1 -- 172.21.15.39:0/2616062971 shutdown_connections 2022-11-27T15:01:51.757 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.740+0000 7fc49f4d6700 1 -- 172.21.15.39:0/2616062971 wait complete. 2022-11-27T15:01:51.758 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.740+0000 7fc49f4d6700 1 Processor -- start 2022-11-27T15:01:51.758 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.740+0000 7fc49f4d6700 1 -- start start 2022-11-27T15:01:51.758 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49f4d6700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49805ccd0 0x7fc498116260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:51.758 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49f4d6700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc49805d6c0 0x7fc4981172d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:51.759 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49f4d6700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc498139b70 0x7fc4980a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:51.759 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49f4d6700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc498116af0 con 0x7fc49805ccd0 2022-11-27T15:01:51.759 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49f4d6700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc498145a90 con 0x7fc49805d6c0 2022-11-27T15:01:51.759 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49f4d6700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc498145d50 con 0x7fc498139b70 2022-11-27T15:01:51.760 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49d272700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49805ccd0 0x7fc498116260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:51.760 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49ca71700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc49805d6c0 0x7fc4981172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:51.760 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49d272700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49805ccd0 0x7fc498116260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51252/0 (socket says 172.21.15.39:51252) 2022-11-27T15:01:51.760 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49d272700 1 -- 172.21.15.39:0/3904414971 learned_addr learned my addr 172.21.15.39:0/3904414971 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:51.760 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49da73700 1 --2- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc498139b70 0x7fc4980a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:51.761 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49d272700 1 -- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc498139b70 msgr2=0x7fc4980a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:51.761 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49d272700 1 --2- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc498139b70 0x7fc4980a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:51.761 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49d272700 1 -- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc49805d6c0 msgr2=0x7fc4981172d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:51.761 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49d272700 1 --2- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc49805d6c0 0x7fc4981172d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:51.762 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49d272700 1 -- 172.21.15.39:0/3904414971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c00b180 con 0x7fc49805ccd0 2022-11-27T15:01:51.762 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.741+0000 7fc49ca71700 1 --2- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc49805d6c0 0x7fc4981172d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:51.762 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.742+0000 7fc49d272700 1 --2- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49805ccd0 0x7fc498116260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fc494002f70 tx=0x7fc494015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:51.762 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.742+0000 7fc48a7fc700 1 -- 172.21.15.39:0/3904414971 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc494011760 con 0x7fc49805ccd0 2022-11-27T15:01:51.762 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.742+0000 7fc48a7fc700 1 -- 172.21.15.39:0/3904414971 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4940118c0 con 0x7fc49805ccd0 2022-11-27T15:01:51.763 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.742+0000 7fc48a7fc700 1 -- 172.21.15.39:0/3904414971 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc494011c20 con 0x7fc49805ccd0 2022-11-27T15:01:51.763 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.742+0000 7fc49f4d6700 1 -- 172.21.15.39:0/3904414971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49400b1e0 con 0x7fc49805ccd0 2022-11-27T15:01:51.763 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.742+0000 7fc49f4d6700 1 -- 172.21.15.39:0/3904414971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4980a68f0 con 0x7fc49805ccd0 2022-11-27T15:01:51.763 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.743+0000 7fc48a7fc700 1 -- 172.21.15.39:0/3904414971 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fc494025030 con 0x7fc49805ccd0 2022-11-27T15:01:51.764 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.743+0000 7fc49f4d6700 1 -- 172.21.15.39:0/3904414971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4980a6160 con 0x7fc49805ccd0 2022-11-27T15:01:51.764 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.743+0000 7fc48a7fc700 1 -- 172.21.15.39:0/3904414971 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3574+0+0 (secure 0 0 0) 0x7fc49404ba50 con 0x7fc49805ccd0 2022-11-27T15:01:51.764 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.746+0000 7fc48a7fc700 1 -- 172.21.15.39:0/3904414971 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fc49404de60 con 0x7fc49805ccd0 2022-11-27T15:01:51.897 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.896+0000 7fc49f4d6700 1 -- 172.21.15.39:0/3904414971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc49805db50 con 0x7fc49805ccd0 2022-11-27T15:01:51.920 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.919+0000 7fc48a7fc700 1 -- 172.21.15.39:0/3904414971 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v5) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc49401e630 con 0x7fc49805ccd0 2022-11-27T15:01:51.928 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.927+0000 7fc49f4d6700 1 -- 172.21.15.39:0/3904414971 >> v1:172.21.15.39:6816/104537 conn(0x7fc478038da0 legacy=0x7fc47803b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:51.928 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.927+0000 7fc49f4d6700 1 -- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49805ccd0 msgr2=0x7fc498116260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:51.928 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.927+0000 7fc49f4d6700 1 --2- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49805ccd0 0x7fc498116260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fc494002f70 tx=0x7fc494015000).stop 2022-11-27T15:01:51.929 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.928+0000 7fc49f4d6700 1 -- 172.21.15.39:0/3904414971 shutdown_connections 2022-11-27T15:01:51.929 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.928+0000 7fc49f4d6700 1 --2- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49805ccd0 0x7fc498116260 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0x7fc494002f70 tx=0x7fc494015000).stop 2022-11-27T15:01:51.929 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.928+0000 7fc49f4d6700 1 --2- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fc498139b70 0x7fc4980a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:51.930 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.928+0000 7fc49f4d6700 1 --2- 172.21.15.39:0/3904414971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc49805d6c0 0x7fc4981172d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:51.930 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.928+0000 7fc49f4d6700 1 -- 172.21.15.39:0/3904414971 >> 172.21.15.39:0/3904414971 conn(0x7fc49806c640 msgr2=0x7fc4980aa960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.929+0000 7fc49f4d6700 1 -- 172.21.15.39:0/3904414971 shutdown_connections 2022-11-27T15:01:51.933 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:51.932+0000 7fc49f4d6700 1 -- 172.21.15.39:0/3904414971 wait complete. 2022-11-27T15:01:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 3 and data pool 4 2022-11-27T15:01:51.964 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T15:01:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.068+0000 7fe438931700 1 Processor -- start 2022-11-27T15:01:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.068+0000 7fe438931700 1 -- start start 2022-11-27T15:01:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe438931700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe434099f40 0x7fe43409b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe438931700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe434133820 0x7fe434140970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe438931700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe434068020 con 0x7fe43409ba10 2022-11-27T15:01:52.078 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe438931700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4340682c0 con 0x7fe434133820 2022-11-27T15:01:52.078 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe438931700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe434067360 con 0x7fe434099f40 2022-11-27T15:01:52.078 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe43259c700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe434099f40 0x7fe43409b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.078 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe432d9d700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe434133820 0x7fe434140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe431d9b700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe43409ba10 0x7fe434133110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51110/0 (socket says 172.21.15.39:51110) 2022-11-27T15:01:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe43259c700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe434099f40 0x7fe43409b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:43700/0 (socket says 172.21.15.39:43700) 2022-11-27T15:01:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe431d9b700 1 -- 172.21.15.39:0/2518699454 learned_addr learned my addr 172.21.15.39:0/2518699454 (peer_addr_for_me v1:172.21.15.39:0/0) 2022-11-27T15:01:52.079 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe43259c700 1 -- 172.21.15.39:0/2518699454 >> v2:172.21.15.39:3300/0 conn(0x7fe434133820 msgr2=0x7fe434140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe43159a700 1 -- 172.21.15.39:0/2518699454 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393483302 0 0) 0x7fe434068020 con 0x7fe43409ba10 2022-11-27T15:01:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe43259c700 1 --2- 172.21.15.39:0/2518699454 >> v2:172.21.15.39:3300/0 conn(0x7fe434133820 0x7fe434140970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe43259c700 1 -- 172.21.15.39:0/2518699454 >> v1:172.21.15.39:6789/0 conn(0x7fe43409ba10 legacy=0x7fe434133110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.080 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.069+0000 7fe43259c700 1 -- 172.21.15.39:0/2518699454 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43405d6a0 con 0x7fe434099f40 2022-11-27T15:01:52.081 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe43259c700 1 --2- 172.21.15.39:0/2518699454 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe434099f40 0x7fe43409b200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fe42c0032f0 tx=0x7fe42c0091d0).ready entity=mon.1 client_cookie=5bdd8471e8d003c8 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:52.081 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe43159a700 1 -- 172.21.15.39:0/2518699454 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe42c00b6c0 con 0x7fe434099f40 2022-11-27T15:01:52.081 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe43159a700 1 -- 172.21.15.39:0/2518699454 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c00b820 con 0x7fe434099f40 2022-11-27T15:01:52.081 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe43159a700 1 -- 172.21.15.39:0/2518699454 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe42c00b990 con 0x7fe434099f40 2022-11-27T15:01:52.081 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe438931700 1 -- 172.21.15.39:0/2518699454 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe434099f40 msgr2=0x7fe43409b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe438931700 1 --2- 172.21.15.39:0/2518699454 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe434099f40 0x7fe43409b200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fe42c0032f0 tx=0x7fe42c0091d0).stop 2022-11-27T15:01:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe438931700 1 -- 172.21.15.39:0/2518699454 shutdown_connections 2022-11-27T15:01:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe438931700 1 --2- 172.21.15.39:0/2518699454 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe434099f40 0x7fe43409b200 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe438931700 1 --2- 172.21.15.39:0/2518699454 >> v2:172.21.15.39:3300/0 conn(0x7fe434133820 0x7fe434140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.083 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe438931700 1 -- 172.21.15.39:0/2518699454 >> 172.21.15.39:0/2518699454 conn(0x7fe43406c410 msgr2=0x7fe43405ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:52.083 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe438931700 1 -- 172.21.15.39:0/2518699454 shutdown_connections 2022-11-27T15:01:52.083 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.070+0000 7fe438931700 1 -- 172.21.15.39:0/2518699454 wait complete. 2022-11-27T15:01:52.083 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.071+0000 7fe438931700 1 Processor -- start 2022-11-27T15:01:52.083 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.071+0000 7fe438931700 1 -- start start 2022-11-27T15:01:52.084 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe438931700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe434099f40 0x7fe4341453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.084 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe438931700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe43409ba10 0x7fe434094cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.084 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe438931700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe434133820 0x7fe4341386c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.084 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe438931700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434098d80 con 0x7fe434099f40 2022-11-27T15:01:52.085 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe438931700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe434145a30 con 0x7fe43409ba10 2022-11-27T15:01:52.085 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe438931700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe434145cf0 con 0x7fe434133820 2022-11-27T15:01:52.085 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe43259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe434099f40 0x7fe4341453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.085 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe43259c700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe434099f40 0x7fe4341453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51262/0 (socket says 172.21.15.39:51262) 2022-11-27T15:01:52.085 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe43259c700 1 -- 172.21.15.39:0/536330458 learned_addr learned my addr 172.21.15.39:0/536330458 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe431d9b700 1 --2- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe43409ba10 0x7fe434094cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.072+0000 7fe432d9d700 1 --2- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe434133820 0x7fe4341386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe43259c700 1 -- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe434133820 msgr2=0x7fe4341386c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe43259c700 1 --2- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe434133820 0x7fe4341386c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe43259c700 1 -- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe43409ba10 msgr2=0x7fe434094cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe43259c700 1 --2- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe43409ba10 0x7fe434094cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe43259c700 1 -- 172.21.15.39:0/536330458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c009470 con 0x7fe434099f40 2022-11-27T15:01:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe431d9b700 1 --2- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe43409ba10 0x7fe434094cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe43259c700 1 --2- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe434099f40 0x7fe4341453a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe42c0032f0 tx=0x7fe42c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe4237fe700 1 -- 172.21.15.39:0/536330458 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe42c00b990 con 0x7fe434099f40 2022-11-27T15:01:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe4237fe700 1 -- 172.21.15.39:0/536330458 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c021630 con 0x7fe434099f40 2022-11-27T15:01:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe438931700 1 -- 172.21.15.39:0/536330458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe434139430 con 0x7fe434099f40 2022-11-27T15:01:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe4237fe700 1 -- 172.21.15.39:0/536330458 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe42c00b6c0 con 0x7fe434099f40 2022-11-27T15:01:52.089 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.073+0000 7fe438931700 1 -- 172.21.15.39:0/536330458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe434138bb0 con 0x7fe434099f40 2022-11-27T15:01:52.089 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.074+0000 7fe4237fe700 1 -- 172.21.15.39:0/536330458 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fe42c027020 con 0x7fe434099f40 2022-11-27T15:01:52.089 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.074+0000 7fe4237fe700 1 -- 172.21.15.39:0/536330458 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3674+0+0 (secure 0 0 0) 0x7fe42c04bd00 con 0x7fe434099f40 2022-11-27T15:01:52.089 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.074+0000 7fe438931700 1 -- 172.21.15.39:0/536330458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe434099900 con 0x7fe434099f40 2022-11-27T15:01:52.090 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.077+0000 7fe4237fe700 1 -- 172.21.15.39:0/536330458 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fe42c015030 con 0x7fe434099f40 2022-11-27T15:01:52.195 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.193+0000 7fe438931700 1 -- 172.21.15.39:0/536330458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe434057a60 con 0x7fe434099f40 2022-11-27T15:01:52.196 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.194+0000 7fe4237fe700 1 -- 172.21.15.39:0/536330458 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+10580 (secure 0 0 0) 0x7fe42c002d20 con 0x7fe434099f40 2022-11-27T15:01:52.196 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:52.196 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":18,"fsid":"fb73c991-5ff5-4983-821e-38ecbf33964f","created":"2022-11-27T15:01:32.796785+0000","modified":"2022-11-27T15:01:51.897593+0000","last_up_change":"2022-11-27T15:01:38.820289+0000","last_in_change":"2022-11-27T15:01:38.820289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-11-27T15:01:34.849418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T15:01:41.877873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-11-27T15:01:50.059993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-11-27T15:01:50.877975+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba0fb473-e933-4ad1-a5f8-2cdd4138f3ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6800","nonce":105137}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6801","nonce":105137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6803","nonce":105137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6802","nonce":105137}]},"public_addr":"172.21.15.39:6800/105137","cluster_addr":"172.21.15.39:6801/105137","heartbeat_back_addr":"172.21.15.39:6803/105137","heartbeat_front_addr":"172.21.15.39:6802/105137","state":["exists","up"]},{"osd":1,"uuid":"d3e5e074-117b-4420-8340-e241cb3e6553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6805","nonce":105140}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6809","nonce":105140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6811","nonce":105140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6810","nonce":105140}]},"public_addr":"172.21.15.39:6805/105140","cluster_addr":"172.21.15.39:6809/105140","heartbeat_back_addr":"172.21.15.39:6811/105140","heartbeat_front_addr":"172.21.15.39:6810/105140","state":["exists","up"]},{"osd":2,"uuid":"2d655d83-8b81-4419-a44f-f4833633c621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6812","nonce":105139}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6813","nonce":105139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6815","nonce":105139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6814","nonce":105139}]},"public_addr":"172.21.15.39:6812/105139","cluster_addr":"172.21.15.39:6813/105139","heartbeat_back_addr":"172.21.15.39:6815/105139","heartbeat_front_addr":"172.21.15.39:6814/105139","state":["exists","up"]},{"osd":3,"uuid":"1f4c9a3a-a40b-43ae-8bd1-7672f6d9a51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6804","nonce":105138}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6806","nonce":105138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6808","nonce":105138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6807","nonce":105138}]},"public_addr":"172.21.15.39:6804/105138","cluster_addr":"172.21.15.39:6806/105138","heartbeat_back_addr":"172.21.15.39:6808/105138","heartbeat_front_addr":"172.21.15.39:6807/105138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.532356+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.825983+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.875364+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.883149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T15:01:52.198 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.196+0000 7fe438931700 1 -- 172.21.15.39:0/536330458 >> v1:172.21.15.39:6816/104537 conn(0x7fe410038dd0 legacy=0x7fe41003b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.199 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.196+0000 7fe438931700 1 -- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe434099f40 msgr2=0x7fe4341453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.199 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.196+0000 7fe438931700 1 --2- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe434099f40 0x7fe4341453a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe42c0032f0 tx=0x7fe42c016000).stop 2022-11-27T15:01:52.199 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.196+0000 7fe438931700 1 -- 172.21.15.39:0/536330458 shutdown_connections 2022-11-27T15:01:52.199 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.196+0000 7fe438931700 1 --2- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe434099f40 0x7fe4341453a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.200 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.196+0000 7fe438931700 1 --2- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7fe434133820 0x7fe4341386c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.200 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.196+0000 7fe438931700 1 --2- 172.21.15.39:0/536330458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe43409ba10 0x7fe434094cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.200 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.196+0000 7fe438931700 1 -- 172.21.15.39:0/536330458 >> 172.21.15.39:0/536330458 conn(0x7fe43406c410 msgr2=0x7fe4340a72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:52.201 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.196+0000 7fe438931700 1 -- 172.21.15.39:0/536330458 shutdown_connections 2022-11-27T15:01:52.201 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.196+0000 7fe438931700 1 -- 172.21.15.39:0/536330458 wait complete. 2022-11-27T15:01:52.208 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T15:01:52.315 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.313+0000 7f80d9e73700 1 Processor -- start 2022-11-27T15:01:52.315 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.313+0000 7f80d9e73700 1 -- start start 2022-11-27T15:01:52.320 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d9e73700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d405be80 0x7f80d405ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d9e73700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d4139f50 0x7f80d405d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d9e73700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d4139620 con 0x7f80d4139b70 2022-11-27T15:01:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d9e73700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d405d1c0 con 0x7f80d4139f50 2022-11-27T15:01:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d9e73700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80d405c2c0 con 0x7f80d405be80 2022-11-27T15:01:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d2ffd700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d405be80 0x7f80d405ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d37fe700 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f80d4139b70 0x7f80d405fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:51116/0 (socket says 172.21.15.39:51116) 2022-11-27T15:01:52.323 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d37fe700 1 -- 172.21.15.39:0/1531751186 learned_addr learned my addr 172.21.15.39:0/1531751186 (peer_addr_for_me v1:172.21.15.39:0/0) 2022-11-27T15:01:52.323 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d3fff700 1 --2- 172.21.15.39:0/1531751186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d4139f50 0x7f80d405d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.323 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d27fc700 1 -- 172.21.15.39:0/1531751186 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475483356 0 0) 0x7f80d4139620 con 0x7f80d4139b70 2022-11-27T15:01:52.323 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d27fc700 1 -- 172.21.15.39:0/1531751186 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80b4003260 con 0x7f80d4139b70 2022-11-27T15:01:52.324 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d3fff700 1 -- 172.21.15.39:0/1531751186 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d405be80 msgr2=0x7f80d405ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:52.324 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d3fff700 1 --2- 172.21.15.39:0/1531751186 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d405be80 0x7f80d405ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.324 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d3fff700 1 -- 172.21.15.39:0/1531751186 >> v1:172.21.15.39:6789/0 conn(0x7f80d4139b70 legacy=0x7f80d405fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.324 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d3fff700 1 -- 172.21.15.39:0/1531751186 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d405dbb0 con 0x7f80d4139f50 2022-11-27T15:01:52.324 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.314+0000 7f80d2ffd700 1 --2- 172.21.15.39:0/1531751186 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d405be80 0x7f80d405ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:52.325 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d3fff700 1 --2- 172.21.15.39:0/1531751186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d4139f50 0x7f80d405d6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f80cc0032f0 tx=0x7f80cc0091d0).ready entity=mon.1 client_cookie=3f46e7ab130c7e48 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:52.325 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d27fc700 1 -- 172.21.15.39:0/1531751186 <== mon.1 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80cc00b640 con 0x7f80d4139f50 2022-11-27T15:01:52.325 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d27fc700 1 -- 172.21.15.39:0/1531751186 <== mon.1 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc00b7a0 con 0x7f80d4139f50 2022-11-27T15:01:52.326 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d27fc700 1 -- 172.21.15.39:0/1531751186 <== mon.1 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80cc00b910 con 0x7f80d4139f50 2022-11-27T15:01:52.326 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d9e73700 1 -- 172.21.15.39:0/1531751186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d4139f50 msgr2=0x7f80d405d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.326 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d9e73700 1 --2- 172.21.15.39:0/1531751186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d4139f50 0x7f80d405d6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f80cc0032f0 tx=0x7f80cc0091d0).stop 2022-11-27T15:01:52.326 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d9e73700 1 -- 172.21.15.39:0/1531751186 shutdown_connections 2022-11-27T15:01:52.326 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d9e73700 1 --2- 172.21.15.39:0/1531751186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d4139f50 0x7f80d405d6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.327 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d9e73700 1 --2- 172.21.15.39:0/1531751186 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d405be80 0x7f80d405ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.327 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d9e73700 1 -- 172.21.15.39:0/1531751186 >> 172.21.15.39:0/1531751186 conn(0x7f80d406c640 msgr2=0x7f80d40ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:52.327 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d9e73700 1 -- 172.21.15.39:0/1531751186 shutdown_connections 2022-11-27T15:01:52.327 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.315+0000 7f80d9e73700 1 -- 172.21.15.39:0/1531751186 wait complete. 2022-11-27T15:01:52.328 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.316+0000 7f80d9e73700 1 Processor -- start 2022-11-27T15:01:52.328 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.316+0000 7f80d9e73700 1 -- start start 2022-11-27T15:01:52.328 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d9e73700 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d405be80 0x7f80d4115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.328 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d9e73700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80d4139b70 0x7f80d4116fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.328 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d9e73700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d4139f50 0x7f80d4149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.329 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d9e73700 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d4148f50 con 0x7f80d4139b70 2022-11-27T15:01:52.329 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d9e73700 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d41141f0 con 0x7f80d405be80 2022-11-27T15:01:52.329 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d9e73700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80d41174c0 con 0x7f80d4139f50 2022-11-27T15:01:52.330 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80d4139b70 0x7f80d4116fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.330 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d2ffd700 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80d4139b70 0x7f80d4116fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51274/0 (socket says 172.21.15.39:51274) 2022-11-27T15:01:52.331 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d3fff700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d4139f50 0x7f80d4149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.331 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d2ffd700 1 -- 172.21.15.39:0/664200481 learned_addr learned my addr 172.21.15.39:0/664200481 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:52.331 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d37fe700 1 --2- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d405be80 0x7f80d4115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.332 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d3fff700 1 -- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d405be80 msgr2=0x7f80d4115e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.332 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d3fff700 1 --2- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d405be80 0x7f80d4115e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.332 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d3fff700 1 -- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80d4139b70 msgr2=0x7f80d4116fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d3fff700 1 --2- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80d4139b70 0x7f80d4116fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d3fff700 1 -- 172.21.15.39:0/664200481 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c400b180 con 0x7f80d4139f50 2022-11-27T15:01:52.333 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d37fe700 1 --2- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d405be80 0x7f80d4115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:52.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.317+0000 7f80d3fff700 1 --2- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d4139f50 0x7f80d4149210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f80cc0032f0 tx=0x7f80cc016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:52.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.318+0000 7f80d0ff9700 1 -- 172.21.15.39:0/664200481 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80cc00b640 con 0x7f80d4139f50 2022-11-27T15:01:52.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.318+0000 7f80d0ff9700 1 -- 172.21.15.39:0/664200481 <== mon.2 v2:172.21.15.39:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc0205f0 con 0x7f80d4139f50 2022-11-27T15:01:52.334 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.318+0000 7f80d9e73700 1 -- 172.21.15.39:0/664200481 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc009470 con 0x7f80d4139f50 2022-11-27T15:01:52.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.318+0000 7f80d0ff9700 1 -- 172.21.15.39:0/664200481 <== mon.2 v2:172.21.15.39:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80cc00bee0 con 0x7f80d4139f50 2022-11-27T15:01:52.335 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.318+0000 7f80d9e73700 1 -- 172.21.15.39:0/664200481 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d4114900 con 0x7f80d4139f50 2022-11-27T15:01:52.336 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.319+0000 7f80d0ff9700 1 -- 172.21.15.39:0/664200481 <== mon.2 v2:172.21.15.39:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f80cc01e070 con 0x7f80d4139f50 2022-11-27T15:01:52.336 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.319+0000 7f80d9e73700 1 -- 172.21.15.39:0/664200481 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d4148cf0 con 0x7f80d4139f50 2022-11-27T15:01:52.337 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.319+0000 7f80d0ff9700 1 -- 172.21.15.39:0/664200481 <== mon.2 v2:172.21.15.39:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3674+0+0 (secure 0 0 0) 0x7f80cc04b2a0 con 0x7f80d4139f50 2022-11-27T15:01:52.338 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.322+0000 7f80d0ff9700 1 -- 172.21.15.39:0/664200481 <== mon.2 v2:172.21.15.39:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f80cc008ba0 con 0x7f80d4139f50 2022-11-27T15:01:52.440 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.439+0000 7f80d9e73700 1 -- 172.21.15.39:0/664200481 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80d413a3e0 con 0x7f80d4139f50 2022-11-27T15:01:52.441 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.439+0000 7f80d0ff9700 1 -- 172.21.15.39:0/664200481 <== mon.2 v2:172.21.15.39:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+10580 (secure 0 0 0) 0x7f80cc025020 con 0x7f80d4139f50 2022-11-27T15:01:52.441 INFO:teuthology.orchestra.run.smithi039.stdout: 2022-11-27T15:01:52.442 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":18,"fsid":"fb73c991-5ff5-4983-821e-38ecbf33964f","created":"2022-11-27T15:01:32.796785+0000","modified":"2022-11-27T15:01:51.897593+0000","last_up_change":"2022-11-27T15:01:38.820289+0000","last_in_change":"2022-11-27T15:01:38.820289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-11-27T15:01:34.849418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T15:01:41.877873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-11-27T15:01:50.059993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-11-27T15:01:50.877975+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"ba0fb473-e933-4ad1-a5f8-2cdd4138f3ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6800","nonce":105137}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6801","nonce":105137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6803","nonce":105137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6802","nonce":105137}]},"public_addr":"172.21.15.39:6800/105137","cluster_addr":"172.21.15.39:6801/105137","heartbeat_back_addr":"172.21.15.39:6803/105137","heartbeat_front_addr":"172.21.15.39:6802/105137","state":["exists","up"]},{"osd":1,"uuid":"d3e5e074-117b-4420-8340-e241cb3e6553","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6805","nonce":105140}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6809","nonce":105140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6811","nonce":105140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6810","nonce":105140}]},"public_addr":"172.21.15.39:6805/105140","cluster_addr":"172.21.15.39:6809/105140","heartbeat_back_addr":"172.21.15.39:6811/105140","heartbeat_front_addr":"172.21.15.39:6810/105140","state":["exists","up"]},{"osd":2,"uuid":"2d655d83-8b81-4419-a44f-f4833633c621","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6812","nonce":105139}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6813","nonce":105139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6815","nonce":105139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6814","nonce":105139}]},"public_addr":"172.21.15.39:6812/105139","cluster_addr":"172.21.15.39:6813/105139","heartbeat_back_addr":"172.21.15.39:6815/105139","heartbeat_front_addr":"172.21.15.39:6814/105139","state":["exists","up"]},{"osd":3,"uuid":"1f4c9a3a-a40b-43ae-8bd1-7672f6d9a51f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6804","nonce":105138}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6806","nonce":105138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6808","nonce":105138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.39:6807","nonce":105138}]},"public_addr":"172.21.15.39:6804/105138","cluster_addr":"172.21.15.39:6806/105138","heartbeat_back_addr":"172.21.15.39:6808/105138","heartbeat_front_addr":"172.21.15.39:6807/105138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.532356+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.825983+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.875364+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T15:01:36.883149+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-11-27T15:01:52.443 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.441+0000 7f80d9e73700 1 -- 172.21.15.39:0/664200481 >> v1:172.21.15.39:6816/104537 conn(0x7f80b8038da0 legacy=0x7f80b803b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.443 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.441+0000 7f80d9e73700 1 -- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d4139f50 msgr2=0x7f80d4149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.444 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.441+0000 7f80d9e73700 1 --2- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d4139f50 0x7f80d4149210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f80cc0032f0 tx=0x7f80cc016000).stop 2022-11-27T15:01:52.444 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.441+0000 7f80d9e73700 1 -- 172.21.15.39:0/664200481 shutdown_connections 2022-11-27T15:01:52.444 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.441+0000 7f80d9e73700 1 --2- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f80d4139f50 0x7f80d4149210 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.444 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.441+0000 7f80d9e73700 1 --2- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f80d405be80 0x7f80d4115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.445 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.441+0000 7f80d9e73700 1 --2- 172.21.15.39:0/664200481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80d4139b70 0x7f80d4116fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.445 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.441+0000 7f80d9e73700 1 -- 172.21.15.39:0/664200481 >> 172.21.15.39:0/664200481 conn(0x7f80d406c640 msgr2=0x7f80d40aa730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T15:01:52.445 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.441+0000 7f80d9e73700 1 -- 172.21.15.39:0/664200481 shutdown_connections 2022-11-27T15:01:52.446 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.441+0000 7f80d9e73700 1 -- 172.21.15.39:0/664200481 wait complete. 2022-11-27T15:01:52.454 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-11-27T15:01:52.564 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.562+0000 7f99d7e9a700 1 Processor -- start 2022-11-27T15:01:52.564 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.563+0000 7f99d7e9a700 1 -- start start 2022-11-27T15:01:52.567 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.563+0000 7f99d7e9a700 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f99d00a3410 0x7f99d0063b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.567 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.563+0000 7f99d7e9a700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f99d00a5040 0x7f99d00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-27T15:01:52.568 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.563+0000 7f99d7e9a700 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d0064000 con 0x7f99d00a3ce0 2022-11-27T15:01:52.568 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.563+0000 7f99d7e9a700 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d0062e90 con 0x7f99d00a3410 2022-11-27T15:01:52.569 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.563+0000 7f99d7e9a700 1 -- --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99d00a4540 con 0x7f99d00a5040 2022-11-27T15:01:52.569 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.563+0000 7f99d5435700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f99d00a5040 0x7f99d00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.569 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.563+0000 7f99d5435700 1 --2- >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f99d00a5040 0x7f99d00a37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.39:3302/0 says I am v2:172.21.15.39:34104/0 (socket says 172.21.15.39:34104) 2022-11-27T15:01:52.569 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.563+0000 7f99d5435700 1 -- 172.21.15.39:0/2370126825 learned_addr learned my addr 172.21.15.39:0/2370126825 (peer_addr_for_me v2:172.21.15.39:0/0) 2022-11-27T15:01:52.569 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.563+0000 7f99d5c36700 1 --2- 172.21.15.39:0/2370126825 >> v2:172.21.15.39:3300/0 conn(0x7f99d00a3410 0x7f99d0063b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-27T15:01:52.570 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.564+0000 7f99d5435700 1 -- 172.21.15.39:0/2370126825 >> v2:172.21.15.39:3300/0 conn(0x7f99d00a3410 msgr2=0x7f99d0063b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T15:01:52.570 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.564+0000 7f99d5435700 1 --2- 172.21.15.39:0/2370126825 >> v2:172.21.15.39:3300/0 conn(0x7f99d00a3410 0x7f99d0063b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-27T15:01:52.570 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.564+0000 7f99d5435700 1 -- 172.21.15.39:0/2370126825 >> v1:172.21.15.39:6789/0 conn(0x7f99d00a3ce0 legacy=0x7f99d01362a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T15:01:52.570 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.564+0000 7f99d5435700 1 -- 172.21.15.39:0/2370126825 --> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d00a2c90 con 0x7f99d00a5040 2022-11-27T15:01:52.571 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.564+0000 7f99d4c34700 1 -- 172.21.15.39:0/2370126825 <== mon.0 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144237787 0 0) 0x7f99c8001490 con 0x7f99d00a3ce0 2022-11-27T15:01:52.571 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.564+0000 7f99d5c36700 1 --2- 172.21.15.39:0/2370126825 >> v2:172.21.15.39:3300/0 conn(0x7f99d00a3410 0x7f99d0063b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-27T15:01:52.571 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.564+0000 7f99d5435700 1 --2- 172.21.15.39:0/2370126825 >> [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] conn(0x7f99d00a5040 0x7f99d00a37f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f99d00a47e0 tx=0x7f99c0006eb0).ready entity=mon.2 client_cookie=5fe64940900ed8c2 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T15:01:52.571 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.564+0000 7f99d4c34700 1 -- 172.21.15.39:0/2370126825 <== mon.2 v2:172.21.15.39:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99c000d5c0 con 0x7f99d00a5040 2022-11-27T15:01:52.571 INFO:teuthology.orchestra.run.smithi039.stderr:2022-11-27T15:01:52.564+0000 7f9