2024-03-13T21:19:42.426 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-13T21:19:42.426 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --verbose --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} --name pdonnell-2024-03-13_20:49:58-fs-wip-batrick-testing-20240310.014948-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-03-13_20:49:58-fs-wip-batrick-testing-20240310.014948-distro-default-smithi/7599086 -- /home/teuthworker/archive/pdonnell-2024-03-13_20:49:58-fs-wip-batrick-testing-20240310.014948-distro-default-smithi/7599086/orig.config.yaml 2024-03-13T21:19:42.500 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T21:19:42.588 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-13_20:49:58-fs-wip-batrick-testing-20240310.014948-distro-default-smithi/7599086 branch: wip-batrick-testing-20240310.014948 description: fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} email: pdonnell@redhat.com first_in_suite: false job_id: '7599086' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi meta: - desc: 'install ceph/octopus latest ' name: pdonnell-2024-03-13_20:49:58-fs-wip-batrick-testing-20240310.014948-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-batrick-testing-20240310.014948 ceph: cephfs: max_mds: 2 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on osd down out interval zero: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request - missing required features sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240310.014948 sha1: 32d4e26ee101db040501914293e54a3c24a95078 owner: scheduled_pdonnell@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - mds.a - mds.b - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - client.0 - - client.1 seed: 6353 sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 sleep_before_teardown: 0 subset: 383/512 suite: fs suite_branch: wip-batrick-testing-20240310.014948 suite_path: /home/teuthworker/src/github.com_batrick_ceph_32d4e26ee101db040501914293e54a3c24a95078/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: 32d4e26ee101db040501914293e54a3c24a95078 targets: smithi023.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB92a+610flmdc9mTv7MjZOowm+0vqk8lSjlxOwWrTCZsijMLF8mQb+hW0QQADandrsYZh/uf/A3xqGpgw5A1MM= smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAls2lVBYvbLoW7nd1zLzZh5uWEuXvqhVopnJeP/7VnUJ8Q03mEv57vINDMw7/mDVEH/jmbsV8HNVp5WiMCAno= smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAi4AMR6ELlSYd3eg2D4jN5YdYwyw4pbrt10cH2HUPI8+eTzjBwLadd5luuvZ3NIUXP1i6idunfDt7CUMwQ8Drk= tasks: - install: branch: octopus exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-rook - ceph-mgr-cephadm - cephadm - ceph-volume extra_packages: - librados2 - print: '**** done installing octopus' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client octopus - print: '**** done ceph' - ceph-fuse: null - print: '**** done octopus client' - workunit: clients: all: - suites/fsstress.sh - print: '**** done fsstress' - mds_pre_upgrade: null - print: '**** done mds pre-upgrade sequence' - install.upgrade: mon.a: branch: quincy - print: '**** done install.upgrade the host' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - ceph.stop: - mds.* - ceph.restart: daemons: - mds.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph osd dump -f json-pretty - ceph versions - ceph osd require-osd-release quincy - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph.healthy: null - print: '**** done ceph.restart' - exec: mon.a: - ceph fs dump --format=json-pretty - ceph fs required_client_features cephfs add metric_collect - sleep: duration: 5 - fs.clients_evicted: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-13_20:49:58 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-13T21:19:42.588 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_32d4e26ee101db040501914293e54a3c24a95078/qa; will attempt to use it 2024-03-13T21:19:42.589 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_32d4e26ee101db040501914293e54a3c24a95078/qa/tasks 2024-03-13T21:19:42.589 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-13T21:19:42.590 INFO:teuthology.task.internal:Checking packages... 2024-03-13T21:19:42.612 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3' 2024-03-13T21:19:42.612 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T21:19:42.612 INFO:teuthology.packaging:ref: None 2024-03-13T21:19:42.612 INFO:teuthology.packaging:tag: None 2024-03-13T21:19:42.612 INFO:teuthology.packaging:branch: wip-batrick-testing-20240310.014948 2024-03-13T21:19:42.612 INFO:teuthology.packaging:sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 2024-03-13T21:19:42.612 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20240310.014948 2024-03-13T21:19:42.755 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2040.gd16d1766 2024-03-13T21:19:42.756 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-13T21:19:42.762 INFO:teuthology.task.internal:no buildpackages task found 2024-03-13T21:19:42.762 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-13T21:19:42.800 INFO:teuthology.task.internal:Saving configuration 2024-03-13T21:19:42.819 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-13T21:19:42.825 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-13T21:19:42.850 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi023.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-13_20:49:58-fs-wip-batrick-testing-20240310.014948-distro-default-smithi/7599086', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 21:13:45.051975', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB92a+610flmdc9mTv7MjZOowm+0vqk8lSjlxOwWrTCZsijMLF8mQb+hW0QQADandrsYZh/uf/A3xqGpgw5A1MM='} 2024-03-13T21:19:42.872 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-13_20:49:58-fs-wip-batrick-testing-20240310.014948-distro-default-smithi/7599086', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 21:13:45.054827', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAls2lVBYvbLoW7nd1zLzZh5uWEuXvqhVopnJeP/7VnUJ8Q03mEv57vINDMw7/mDVEH/jmbsV8HNVp5WiMCAno='} 2024-03-13T21:19:42.895 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-13_20:49:58-fs-wip-batrick-testing-20240310.014948-distro-default-smithi/7599086', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 21:13:45.053837', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAi4AMR6ELlSYd3eg2D4jN5YdYwyw4pbrt10cH2HUPI8+eTzjBwLadd5luuvZ3NIUXP1i6idunfDt7CUMwQ8Drk='} 2024-03-13T21:19:42.896 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-13T21:19:42.901 INFO:teuthology.task.internal:roles: ubuntu@smithi023.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2024-03-13T21:19:42.901 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['client.0'] 2024-03-13T21:19:42.901 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['client.1'] 2024-03-13T21:19:42.901 INFO:teuthology.run_tasks:Running task console_log... 2024-03-13T21:19:43.055 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3a5fcfd160>, signals=[15]) 2024-03-13T21:19:43.056 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-13T21:19:43.061 INFO:teuthology.task.internal:Opening connections... 2024-03-13T21:19:43.061 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi023.front.sepia.ceph.com 2024-03-13T21:19:43.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:19:43.136 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-03-13T21:19:43.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:19:43.208 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-03-13T21:19:43.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:19:43.282 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-13T21:19:43.290 DEBUG:teuthology.orchestra.run.smithi023:> uname -m 2024-03-13T21:19:43.313 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64 2024-03-13T21:19:43.313 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release 2024-03-13T21:19:43.375 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream" 2024-03-13T21:19:43.375 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="8" 2024-03-13T21:19:43.375 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2024-03-13T21:19:43.375 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2024-03-13T21:19:43.375 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="8" 2024-03-13T21:19:43.375 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el8" 2024-03-13T21:19:43.375 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-13T21:19:43.376 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2024-03-13T21:19:43.376 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-13T21:19:43.376 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/" 2024-03-13T21:19:43.376 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T21:19:43.376 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-13T21:19:43.376 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T21:19:43.376 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2024-03-13T21:19:43.401 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-03-13T21:19:43.423 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-03-13T21:19:43.424 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-03-13T21:19:43.484 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-03-13T21:19:43.484 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8" 2024-03-13T21:19:43.484 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-03-13T21:19:43.485 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-03-13T21:19:43.485 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8" 2024-03-13T21:19:43.485 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2024-03-13T21:19:43.485 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-13T21:19:43.485 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-03-13T21:19:43.485 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-13T21:19:43.485 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-03-13T21:19:43.485 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T21:19:43.485 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-13T21:19:43.485 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T21:19:43.485 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-03-13T21:19:43.509 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-03-13T21:19:43.531 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-03-13T21:19:43.531 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-03-13T21:19:43.592 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-03-13T21:19:43.592 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="8" 2024-03-13T21:19:43.592 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-03-13T21:19:43.592 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-03-13T21:19:43.592 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="8" 2024-03-13T21:19:43.592 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el8" 2024-03-13T21:19:43.592 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-13T21:19:43.592 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-03-13T21:19:43.593 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-13T21:19:43.593 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-03-13T21:19:43.593 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T21:19:43.593 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-13T21:19:43.593 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T21:19:43.593 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-03-13T21:19:43.617 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-13T21:19:43.625 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-13T21:19:43.641 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-13T21:19:43.641 DEBUG:teuthology.orchestra.run.smithi023:> test '!' -e /home/ubuntu/cephtest 2024-03-13T21:19:43.644 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-03-13T21:19:43.646 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-03-13T21:19:43.666 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-13T21:19:43.671 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-13T21:19:43.671 DEBUG:teuthology.orchestra.run.smithi023:> test -z $(ls -A /var/lib/ceph) 2024-03-13T21:19:43.705 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-03-13T21:19:43.707 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-03-13T21:19:43.741 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-13T21:19:43.817 INFO:teuthology.run_tasks:Running task kernel... 2024-03-13T21:19:43.833 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-13T21:19:43.833 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-13T21:19:43.834 DEBUG:teuthology.orchestra.run.smithi023:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-13T21:19:43.835 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-13T21:19:43.835 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-13T21:19:43.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T21:19:43.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T21:19:43.858 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-03-13T21:19:43.858 DEBUG:teuthology.orchestra.run.smithi023:> uname -r 2024-03-13T21:19:43.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T21:19:43.859 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-03-13T21:19:43.918 INFO:teuthology.orchestra.run.smithi077.stdout:4.18.0-489.el8.x86_64 2024-03-13T21:19:43.918 INFO:teuthology.task.kernel:Running kernel on smithi077: 4.18.0-489.el8.x86_64 2024-03-13T21:19:43.919 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum install -y kernel 2024-03-13T21:19:43.920 INFO:teuthology.orchestra.run.smithi023.stdout:4.18.0-489.el8.x86_64 2024-03-13T21:19:43.920 INFO:teuthology.task.kernel:Running kernel on smithi023: 4.18.0-489.el8.x86_64 2024-03-13T21:19:43.921 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum install -y kernel 2024-03-13T21:19:43.922 INFO:teuthology.orchestra.run.smithi064.stdout:4.18.0-489.el8.x86_64 2024-03-13T21:19:43.922 INFO:teuthology.task.kernel:Running kernel on smithi064: 4.18.0-489.el8.x86_64 2024-03-13T21:19:43.922 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-03-13T21:19:47.340 INFO:teuthology.orchestra.run.smithi023.stdout:CentOS Stream 8 - AppStream 34 MB/s | 28 MB 00:00 2024-03-13T21:19:47.371 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 8 - AppStream 33 MB/s | 28 MB 00:00 2024-03-13T21:19:47.525 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - AppStream 30 MB/s | 28 MB 00:00 2024-03-13T21:19:53.439 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 8 - BaseOS 9.3 MB/s | 10 MB 00:01 2024-03-13T21:19:53.857 INFO:teuthology.orchestra.run.smithi023.stdout:CentOS Stream 8 - BaseOS 8.4 MB/s | 10 MB 00:01 2024-03-13T21:19:53.958 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - BaseOS 9.3 MB/s | 10 MB 00:01 2024-03-13T21:19:55.656 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 8 - Extras 33 kB/s | 18 kB 00:00 2024-03-13T21:19:55.990 INFO:teuthology.orchestra.run.smithi023.stdout:CentOS Stream 8 - Extras 46 kB/s | 18 kB 00:00 2024-03-13T21:19:56.015 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 8 - Extras common packages 29 kB/s | 7.2 kB 00:00 2024-03-13T21:19:56.208 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras 35 kB/s | 18 kB 00:00 2024-03-13T21:19:56.249 INFO:teuthology.orchestra.run.smithi077.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-03-13T21:19:56.366 INFO:teuthology.orchestra.run.smithi023.stdout:CentOS Stream 8 - Extras common packages 28 kB/s | 7.2 kB 00:00 2024-03-13T21:19:56.576 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras common packages 29 kB/s | 7.2 kB 00:00 2024-03-13T21:19:56.635 INFO:teuthology.orchestra.run.smithi023.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2024-03-13T21:19:56.811 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-03-13T21:19:56.959 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 26 MB/s | 16 MB 00:00 2024-03-13T21:20:00.862 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 262 kB/s | 24 kB 00:00 2024-03-13T21:20:02.577 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 2.8 MB/s | 16 MB 00:05 2024-03-13T21:20:03.901 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-03-13T21:20:03.901 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-03-13T21:20:04.030 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:20:04.031 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:20:04.031 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-03-13T21:20:04.031 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:20:04.031 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:20:04.031 INFO:teuthology.orchestra.run.smithi077.stdout: kernel x86_64 4.18.0-544.el8 baseos 10 M 2024-03-13T21:20:04.031 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-03-13T21:20:04.031 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-core x86_64 4.18.0-544.el8 baseos 44 M 2024-03-13T21:20:04.031 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules x86_64 4.18.0-544.el8 baseos 36 M 2024-03-13T21:20:04.032 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:20:04.032 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:20:04.032 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:20:04.032 INFO:teuthology.orchestra.run.smithi077.stdout:Install 3 Packages 2024-03-13T21:20:04.032 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:20:04.032 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 90 M 2024-03-13T21:20:04.033 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 97 M 2024-03-13T21:20:04.033 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:20:05.086 INFO:teuthology.orchestra.run.smithi077.stdout:(1/3): kernel-4.18.0-544.el8.x86_64.rpm 10 MB/s | 10 MB 00:01 2024-03-13T21:20:06.136 INFO:teuthology.orchestra.run.smithi077.stdout:(2/3): kernel-modules-4.18.0-544.el8.x86_64.rpm 18 MB/s | 36 MB 00:02 2024-03-13T21:20:06.561 INFO:teuthology.orchestra.run.smithi077.stdout:(3/3): kernel-core-4.18.0-544.el8.x86_64.rpm 18 MB/s | 44 MB 00:02 2024-03-13T21:20:06.562 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:20:06.562 INFO:teuthology.orchestra.run.smithi077.stdout:Total 36 MB/s | 90 MB 00:02 2024-03-13T21:20:06.620 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 407 kB/s | 24 kB 00:00 2024-03-13T21:20:07.244 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:20:07.650 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:20:07.651 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:20:08.891 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:20:08.914 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:20:09.808 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-03-13T21:20:09.808 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-03-13T21:20:09.919 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:20:09.919 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 4.18.0-544.el8 baseos 10 M 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 4.18.0-544.el8 baseos 44 M 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 4.18.0-544.el8 baseos 36 M 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout:Install 3 Packages 2024-03-13T21:20:09.920 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:20:09.921 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 90 M 2024-03-13T21:20:09.921 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 97 M 2024-03-13T21:20:09.921 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:20:11.196 INFO:teuthology.orchestra.run.smithi064.stdout:(1/3): kernel-4.18.0-544.el8.x86_64.rpm 9.5 MB/s | 10 MB 00:01 2024-03-13T21:20:11.735 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:20:12.188 INFO:teuthology.orchestra.run.smithi064.stdout:(2/3): kernel-modules-4.18.0-544.el8.x86_64.rpm 17 MB/s | 36 MB 00:02 2024-03-13T21:20:12.224 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-core-4.18.0-544.el8.x86_64 1/3 2024-03-13T21:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout:(3/3): kernel-core-4.18.0-544.el8.x86_64.rpm 17 MB/s | 44 MB 00:02 2024-03-13T21:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:20:12.680 INFO:teuthology.orchestra.run.smithi064.stdout:Total 33 MB/s | 90 MB 00:02 2024-03-13T21:20:13.398 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:20:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:20:13.838 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:20:15.144 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:20:15.160 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:20:15.783 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-core-4.18.0-544.el8.x86_64 1/3 2024-03-13T21:20:16.215 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-modules-4.18.0-544.el8.x86_64 2/3 2024-03-13T21:20:17.008 INFO:teuthology.orchestra.run.smithi023.stdout:Extra Packages for Enterprise Linux 816 kB/s | 16 MB 00:20 2024-03-13T21:20:18.121 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:20:18.613 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-4.18.0-544.el8.x86_64 1/3 2024-03-13T21:20:19.663 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-4.18.0-544.el8.x86_64 2/3 2024-03-13T21:20:19.973 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:21.108 INFO:teuthology.orchestra.run.smithi023.stdout:lab-extras 504 kB/s | 24 kB 00:00 2024-03-13T21:20:22.424 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-4.18.0-544.el8.x86_64 1/3 2024-03-13T21:20:22.813 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-4.18.0-544.el8.x86_64 2/3 2024-03-13T21:20:24.464 INFO:teuthology.orchestra.run.smithi023.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-03-13T21:20:24.464 INFO:teuthology.orchestra.run.smithi023.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-03-13T21:20:24.580 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:20:24.581 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:20:24.581 INFO:teuthology.orchestra.run.smithi023.stdout: Package Architecture Version Repository Size 2024-03-13T21:20:24.581 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:20:24.581 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:20:24.581 INFO:teuthology.orchestra.run.smithi023.stdout: kernel x86_64 4.18.0-544.el8 baseos 10 M 2024-03-13T21:20:24.581 INFO:teuthology.orchestra.run.smithi023.stdout:Installing dependencies: 2024-03-13T21:20:24.581 INFO:teuthology.orchestra.run.smithi023.stdout: kernel-core x86_64 4.18.0-544.el8 baseos 44 M 2024-03-13T21:20:24.581 INFO:teuthology.orchestra.run.smithi023.stdout: kernel-modules x86_64 4.18.0-544.el8 baseos 36 M 2024-03-13T21:20:24.581 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:20:24.581 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:20:24.582 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:20:24.582 INFO:teuthology.orchestra.run.smithi023.stdout:Install 3 Packages 2024-03-13T21:20:24.582 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:20:24.582 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 90 M 2024-03-13T21:20:24.582 INFO:teuthology.orchestra.run.smithi023.stdout:Installed size: 97 M 2024-03-13T21:20:24.582 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:20:26.243 INFO:teuthology.orchestra.run.smithi023.stdout:(1/3): kernel-4.18.0-544.el8.x86_64.rpm 9.5 MB/s | 10 MB 00:01 2024-03-13T21:20:26.494 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-4.18.0-544.el8.x86_64 2/3 2024-03-13T21:20:27.050 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:27.351 INFO:teuthology.orchestra.run.smithi023.stdout:(2/3): kernel-modules-4.18.0-544.el8.x86_64.rpm 17 MB/s | 36 MB 00:02 2024-03-13T21:20:27.910 INFO:teuthology.orchestra.run.smithi023.stdout:(3/3): kernel-core-4.18.0-544.el8.x86_64.rpm 16 MB/s | 44 MB 00:02 2024-03-13T21:20:27.910 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:20:27.910 INFO:teuthology.orchestra.run.smithi023.stdout:Total 27 MB/s | 90 MB 00:03 2024-03-13T21:20:28.634 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:20:29.097 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:20:29.097 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:20:30.415 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:20:30.441 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:20:33.373 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:20:33.874 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : kernel-core-4.18.0-544.el8.x86_64 1/3 2024-03-13T21:20:37.600 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: kernel-core-4.18.0-544.el8.x86_64 1/3 2024-03-13T21:20:38.007 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : kernel-modules-4.18.0-544.el8.x86_64 2/3 2024-03-13T21:20:41.708 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: kernel-modules-4.18.0-544.el8.x86_64 2/3 2024-03-13T21:20:41.867 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-core-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:41.891 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:42.020 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : kernel-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:42.234 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:42.234 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-4.18.0-544.el8.x86_64 1/3 2024-03-13T21:20:42.234 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-core-4.18.0-544.el8.x86_64 2/3 2024-03-13T21:20:42.943 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-modules-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:20:42.944 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:20:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-4.18.0-544.el8.x86_64 kernel-core-4.18.0-544.el8.x86_64 2024-03-13T21:20:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules-4.18.0-544.el8.x86_64 2024-03-13T21:20:42.944 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:20:42.944 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:20:43.392 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel | sort -rV | head -n 1 2024-03-13T21:20:43.614 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-4.18.0-544.el8.x86_64 2024-03-13T21:20:43.614 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-544.el8.x86_64 2024-03-13T21:20:43.614 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-544.el8.x86_64 2024-03-13T21:20:43.614 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-13T21:20:43.614 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-13T21:20:43.615 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-13T21:20:43.615 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi077.front.sepia.ceph.com, path=None, version=distro) 2024-03-13T21:20:43.615 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel | sort -rV | head -n 1 2024-03-13T21:20:43.825 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-4.18.0-544.el8.x86_64 2024-03-13T21:20:43.825 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-544.el8.x86_64 2024-03-13T21:20:43.825 DEBUG:teuthology.orchestra.run.smithi077:> sudo rpm -qi grub2-tools 2024-03-13T21:20:43.860 INFO:teuthology.orchestra.run.smithi077.stdout:Name : grub2-tools 2024-03-13T21:20:43.860 INFO:teuthology.orchestra.run.smithi077.stdout:Epoch : 1 2024-03-13T21:20:43.860 INFO:teuthology.orchestra.run.smithi077.stdout:Version : 2.02 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Release : 129.el8 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Architecture: x86_64 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Group : System Environment/Base 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Size : 9265586 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:License : GPLv3+ 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Build Host : kbuilder.bsys.centos.org 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Relocations : (not relocatable) 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Packager : CentOS BuildSystem 2024-03-13T21:20:43.861 INFO:teuthology.orchestra.run.smithi077.stdout:Vendor : CentOS 2024-03-13T21:20:43.862 INFO:teuthology.orchestra.run.smithi077.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-13T21:20:43.862 INFO:teuthology.orchestra.run.smithi077.stdout:Summary : Support tools for GRUB. 2024-03-13T21:20:43.862 INFO:teuthology.orchestra.run.smithi077.stdout:Description : 2024-03-13T21:20:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:20:43.862 INFO:teuthology.orchestra.run.smithi077.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-13T21:20:43.862 INFO:teuthology.orchestra.run.smithi077.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-13T21:20:43.862 INFO:teuthology.orchestra.run.smithi077.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-13T21:20:43.862 INFO:teuthology.orchestra.run.smithi077.stdout:hardware devices. 2024-03-13T21:20:43.862 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:20:43.862 INFO:teuthology.orchestra.run.smithi077.stdout:This subpackage provides tools for support of all platforms. 2024-03-13T21:20:43.863 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-13T21:20:43.863 INFO:teuthology.task.kernel:Updating grub on smithi077 to boot 4.18.0-544.el8.x86_64 2024-03-13T21:20:43.863 DEBUG:teuthology.orchestra.run.smithi077:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-13T21:20:45.218 INFO:teuthology.orchestra.run.smithi077.stderr:Generating grub configuration file ... 2024-03-13T21:20:47.833 INFO:teuthology.orchestra.run.smithi077.stderr:done 2024-03-13T21:20:47.835 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2024-03-13T21:20:47.873 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.zURgSBIepP 2024-03-13T21:20:47.874 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zURgSBIepP 2024-03-13T21:20:47.947 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.zURgSBIepP 2024-03-13T21:20:48.088 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.zURgSBIepP is 6KB 2024-03-13T21:20:48.136 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.zURgSBIepP 2024-03-13T21:20:48.153 DEBUG:teuthology.orchestra.run.smithi077:> sudo /bin/ls /boot/loader/entries || true 2024-03-13T21:20:48.221 INFO:teuthology.orchestra.run.smithi077.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-03-13T21:20:48.235 INFO:teuthology.orchestra.run.smithi077.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-03-13T21:20:48.235 INFO:teuthology.orchestra.run.smithi077.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-03-13T21:20:48.235 INFO:teuthology.orchestra.run.smithi077.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-544.el8.x86_64.conf 2024-03-13T21:20:48.236 DEBUG:teuthology.orchestra.run.smithi077:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-544.el8.x86_64 2024-03-13T21:20:48.413 DEBUG:teuthology.orchestra.run.smithi077:> sudo shutdown -r now 2024-03-13T21:20:50.414 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:50.438 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:50.806 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:50.806 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-4.18.0-544.el8.x86_64 1/3 2024-03-13T21:20:50.806 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-4.18.0-544.el8.x86_64 2/3 2024-03-13T21:20:51.341 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:20:51.341 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:20:51.341 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:20:51.341 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-4.18.0-544.el8.x86_64 kernel-core-4.18.0-544.el8.x86_64 2024-03-13T21:20:51.341 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-4.18.0-544.el8.x86_64 2024-03-13T21:20:51.341 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:20:51.341 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:20:51.479 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-03-13T21:20:51.746 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-4.18.0-544.el8.x86_64 2024-03-13T21:20:51.746 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-544.el8.x86_64 2024-03-13T21:20:51.746 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-544.el8.x86_64 2024-03-13T21:20:51.746 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-13T21:20:51.746 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-13T21:20:51.746 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-13T21:20:51.747 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-03-13T21:20:51.747 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-03-13T21:20:51.966 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-4.18.0-544.el8.x86_64 2024-03-13T21:20:51.967 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-544.el8.x86_64 2024-03-13T21:20:51.967 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-03-13T21:20:52.001 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-03-13T21:20:52.001 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-03-13T21:20:52.001 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.02 2024-03-13T21:20:52.001 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 129.el8 2024-03-13T21:20:52.001 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-03-13T21:20:52.001 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:Group : System Environment/Base 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 9265586 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : kbuilder.bsys.centos.org 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:Relocations : (not relocatable) 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : CentOS BuildSystem 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-03-13T21:20:52.002 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-13T21:20:52.003 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-03-13T21:20:52.003 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-03-13T21:20:52.003 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:20:52.003 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-13T21:20:52.003 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-13T21:20:52.003 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-13T21:20:52.003 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-03-13T21:20:52.003 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:20:52.003 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-03-13T21:20:52.004 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-13T21:20:52.004 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 4.18.0-544.el8.x86_64 2024-03-13T21:20:52.005 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-13T21:20:53.686 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-03-13T21:20:55.932 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-03-13T21:20:55.933 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-03-13T21:20:55.964 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.qQLQson1N5 2024-03-13T21:20:55.964 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qQLQson1N5 2024-03-13T21:20:56.051 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.qQLQson1N5 2024-03-13T21:20:56.150 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.qQLQson1N5 is 6KB 2024-03-13T21:20:56.198 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.qQLQson1N5 2024-03-13T21:20:56.216 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-03-13T21:20:56.281 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-03-13T21:20:56.282 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-03-13T21:20:56.282 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-03-13T21:20:56.282 INFO:teuthology.orchestra.run.smithi064.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-544.el8.x86_64.conf 2024-03-13T21:20:56.283 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-544.el8.x86_64 2024-03-13T21:20:56.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-03-13T21:21:05.413 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: kernel-core-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:21:05.435 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: kernel-modules-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:21:05.802 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: kernel-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:21:05.802 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : kernel-4.18.0-544.el8.x86_64 1/3 2024-03-13T21:21:05.802 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : kernel-core-4.18.0-544.el8.x86_64 2/3 2024-03-13T21:21:06.716 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : kernel-modules-4.18.0-544.el8.x86_64 3/3 2024-03-13T21:21:06.716 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:21:06.716 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:21:06.716 INFO:teuthology.orchestra.run.smithi023.stdout: kernel-4.18.0-544.el8.x86_64 kernel-core-4.18.0-544.el8.x86_64 2024-03-13T21:21:06.716 INFO:teuthology.orchestra.run.smithi023.stdout: kernel-modules-4.18.0-544.el8.x86_64 2024-03-13T21:21:06.716 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:21:06.716 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:21:06.861 DEBUG:teuthology.orchestra.run.smithi023:> rpm -q kernel | sort -rV | head -n 1 2024-03-13T21:21:07.090 INFO:teuthology.orchestra.run.smithi023.stdout:kernel-4.18.0-544.el8.x86_64 2024-03-13T21:21:07.091 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-544.el8.x86_64 2024-03-13T21:21:07.092 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-544.el8.x86_64 2024-03-13T21:21:07.092 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-13T21:21:07.092 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-13T21:21:07.092 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-13T21:21:07.092 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi023.front.sepia.ceph.com, path=None, version=distro) 2024-03-13T21:21:07.092 DEBUG:teuthology.orchestra.run.smithi023:> rpm -q kernel | sort -rV | head -n 1 2024-03-13T21:21:07.312 INFO:teuthology.orchestra.run.smithi023.stdout:kernel-4.18.0-544.el8.x86_64 2024-03-13T21:21:07.312 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-544.el8.x86_64 2024-03-13T21:21:07.312 DEBUG:teuthology.orchestra.run.smithi023:> sudo rpm -qi grub2-tools 2024-03-13T21:21:07.347 INFO:teuthology.orchestra.run.smithi023.stdout:Name : grub2-tools 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Epoch : 1 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Version : 2.02 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Release : 129.el8 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Architecture: x86_64 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Group : System Environment/Base 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Size : 9265586 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:License : GPLv3+ 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Build Host : kbuilder.bsys.centos.org 2024-03-13T21:21:07.348 INFO:teuthology.orchestra.run.smithi023.stdout:Relocations : (not relocatable) 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout:Packager : CentOS BuildSystem 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout:Vendor : CentOS 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout:Summary : Support tools for GRUB. 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout:Description : 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout:hardware devices. 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:21:07.349 INFO:teuthology.orchestra.run.smithi023.stdout:This subpackage provides tools for support of all platforms. 2024-03-13T21:21:07.350 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-13T21:21:07.350 INFO:teuthology.task.kernel:Updating grub on smithi023 to boot 4.18.0-544.el8.x86_64 2024-03-13T21:21:07.350 DEBUG:teuthology.orchestra.run.smithi023:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-13T21:21:08.667 INFO:teuthology.orchestra.run.smithi023.stderr:Generating grub configuration file ... 2024-03-13T21:21:10.894 INFO:teuthology.orchestra.run.smithi023.stderr:done 2024-03-13T21:21:10.896 DEBUG:teuthology.orchestra.run.smithi023:> mktemp 2024-03-13T21:21:10.939 INFO:teuthology.orchestra.run.smithi023.stdout:/tmp/tmp.bYoB6wwYUI 2024-03-13T21:21:10.939 DEBUG:teuthology.orchestra.run.smithi023:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bYoB6wwYUI 2024-03-13T21:21:11.016 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod 0666 /tmp/tmp.bYoB6wwYUI 2024-03-13T21:21:11.185 DEBUG:teuthology.orchestra.remote:smithi023:/tmp/tmp.bYoB6wwYUI is 6KB 2024-03-13T21:21:11.233 DEBUG:teuthology.orchestra.run.smithi023:> rm -fr /tmp/tmp.bYoB6wwYUI 2024-03-13T21:21:11.251 DEBUG:teuthology.orchestra.run.smithi023:> sudo /bin/ls /boot/loader/entries || true 2024-03-13T21:21:11.331 INFO:teuthology.orchestra.run.smithi023.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-03-13T21:21:11.331 INFO:teuthology.orchestra.run.smithi023.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-03-13T21:21:11.331 INFO:teuthology.orchestra.run.smithi023.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-03-13T21:21:11.331 INFO:teuthology.orchestra.run.smithi023.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-544.el8.x86_64.conf 2024-03-13T21:21:11.332 DEBUG:teuthology.orchestra.run.smithi023:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-544.el8.x86_64 2024-03-13T21:21:11.480 DEBUG:teuthology.orchestra.run.smithi023:> sudo shutdown -r now 2024-03-13T21:21:18.422 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-13T21:21:18.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-03-13T21:21:18.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:21:26.438 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-13T21:21:26.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-13T21:21:26.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:21:36.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.77 2024-03-13T21:21:41.486 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-13T21:21:41.487 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-03-13T21:21:41.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:21:44.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-03-13T21:21:45.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-03-13T21:21:45.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:21:48.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.77 2024-03-13T21:21:53.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-13T21:21:53.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:21:57.058 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-03-13T21:21:59.842 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-03-13T21:22:00.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-03-13T21:22:00.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:22:08.851 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-03-13T21:22:08.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:22:09.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-13T21:22:09.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:22:11.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-03-13T21:22:23.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-03-13T21:22:23.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:22:27.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-03-13T21:22:27.490 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-03-13T21:22:33.249 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-03-13T21:22:33.873 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-03-13T21:22:33.873 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-544.el8.x86_64"... 2024-03-13T21:22:33.873 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-03-13T21:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout:4.18.0-544.el8.x86_64 2024-03-13T21:22:33.892 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-544.el8.x86_64 vs 4.18.0-544.el8.x86_64 2024-03-13T21:22:33.892 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-13T21:22:33.892 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-13T21:22:34.893 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2024-03-13T21:22:34.893 DEBUG:teuthology.orchestra.run.smithi077:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-13T21:22:34.955 INFO:teuthology.orchestra.run.smithi077.stdout:ttyS1 2024-03-13T21:22:34.970 DEBUG:teuthology.parallel:result is None 2024-03-13T21:22:42.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-03-13T21:22:42.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:22:42.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-13T21:22:42.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:22:43.258 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-03-13T21:22:43.748 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-13T21:22:43.748 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-544.el8.x86_64"... 2024-03-13T21:22:43.749 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-03-13T21:22:43.766 INFO:teuthology.orchestra.run.smithi064.stdout:4.18.0-544.el8.x86_64 2024-03-13T21:22:43.766 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-544.el8.x86_64 vs 4.18.0-544.el8.x86_64 2024-03-13T21:22:43.766 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-13T21:22:43.766 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-13T21:22:44.767 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-03-13T21:22:44.767 DEBUG:teuthology.orchestra.run.smithi064:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-13T21:22:44.834 INFO:teuthology.orchestra.run.smithi064.stdout:ttyS1 2024-03-13T21:22:44.849 DEBUG:teuthology.parallel:result is None 2024-03-13T21:23:00.514 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-03-13T21:23:18.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-03-13T21:23:18.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:23:18.861 DEBUG:teuthology.orchestra.run.smithi023:> true 2024-03-13T21:23:19.103 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-03-13T21:23:19.104 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-544.el8.x86_64"... 2024-03-13T21:23:19.104 DEBUG:teuthology.orchestra.run.smithi023:> uname -r 2024-03-13T21:23:19.121 INFO:teuthology.orchestra.run.smithi023.stdout:4.18.0-544.el8.x86_64 2024-03-13T21:23:19.121 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-544.el8.x86_64 vs 4.18.0-544.el8.x86_64 2024-03-13T21:23:19.121 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-13T21:23:19.122 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-13T21:23:20.122 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-13T21:23:20.123 DEBUG:teuthology.orchestra.run.smithi023:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-13T21:23:20.273 INFO:teuthology.orchestra.run.smithi023.stdout:ttyS1 2024-03-13T21:23:20.309 DEBUG:teuthology.parallel:result is None 2024-03-13T21:23:20.309 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-13T21:23:20.316 INFO:teuthology.task.internal:Creating test directory... 2024-03-13T21:23:20.316 DEBUG:teuthology.orchestra.run.smithi023:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-13T21:23:20.338 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-13T21:23:20.341 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-13T21:23:20.368 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-13T21:23:20.376 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-13T21:23:20.426 INFO:teuthology.task.internal:Creating archive directory... 2024-03-13T21:23:20.426 DEBUG:teuthology.orchestra.run.smithi023:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-13T21:23:20.429 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-13T21:23:20.431 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-13T21:23:20.646 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-13T21:23:20.653 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-13T21:23:20.654 DEBUG:teuthology.orchestra.run.smithi023:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-13T21:23:20.702 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-13T21:23:20.704 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-13T21:23:20.739 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T21:23:20.750 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T21:23:20.750 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T21:23:20.762 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T21:23:20.933 INFO:teuthology.orchestra.run.smithi023.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T21:23:21.052 INFO:teuthology.orchestra.run.smithi023.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T21:23:21.052 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-13T21:23:21.061 INFO:teuthology.task.internal:Configuring sudo... 2024-03-13T21:23:21.061 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-13T21:23:21.104 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-13T21:23:21.106 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-13T21:23:21.355 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-13T21:23:21.366 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-13T21:23:21.366 DEBUG:teuthology.orchestra.run.smithi023:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-13T21:23:21.411 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-13T21:23:21.414 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-13T21:23:21.681 DEBUG:teuthology.orchestra.run.smithi023:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T21:23:22.078 DEBUG:teuthology.orchestra.run.smithi023:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T21:23:22.119 DEBUG:teuthology.orchestra.run.smithi023:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T21:23:22.205 DEBUG:teuthology.orchestra.run.smithi023:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T21:23:22.249 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:23:22.249 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-13T21:23:22.317 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T21:23:22.369 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T21:23:22.430 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T21:23:22.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T21:23:22.524 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-13T21:23:22.524 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-13T21:23:22.595 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T21:23:22.643 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T21:23:22.700 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T21:23:22.786 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T21:23:22.830 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-03-13T21:23:22.830 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-13T21:23:22.899 DEBUG:teuthology.orchestra.run.smithi023:> sudo service rsyslog restart 2024-03-13T21:23:22.901 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-03-13T21:23:22.903 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-03-13T21:23:22.964 INFO:teuthology.orchestra.run.smithi023.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T21:23:22.964 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T21:23:23.008 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T21:23:23.396 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-13T21:23:23.405 INFO:teuthology.task.internal:Starting timer... 2024-03-13T21:23:23.405 INFO:teuthology.run_tasks:Running task pcp... 2024-03-13T21:23:23.441 INFO:teuthology.run_tasks:Running task selinux... 2024-03-13T21:23:23.495 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-13T21:23:23.496 DEBUG:teuthology.orchestra.run.smithi023:> sudo service auditd rotate 2024-03-13T21:23:23.576 INFO:teuthology.orchestra.run.smithi023.stdout:Rotating logs: 2024-03-13T21:23:23.577 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-03-13T21:23:23.685 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-03-13T21:23:23.687 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-03-13T21:23:23.759 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-03-13T21:23:23.760 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-13T21:23:23.761 DEBUG:teuthology.orchestra.run.smithi023:> /usr/sbin/getenforce 2024-03-13T21:23:23.823 INFO:teuthology.orchestra.run.smithi023.stdout:Permissive 2024-03-13T21:23:23.824 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-03-13T21:23:23.886 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-03-13T21:23:23.886 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-03-13T21:23:23.928 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-03-13T21:23:23.928 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi023.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi077.front.sepia.ceph.com': 'permissive'} 2024-03-13T21:23:23.928 DEBUG:teuthology.orchestra.run.smithi023:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-13T21:23:23.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T21:23:23.956 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-13T21:23:23.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T21:23:23.986 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-13T21:23:24.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T21:23:24.013 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-13T21:23:24.013 DEBUG:teuthology.orchestra.run.smithi023:> sudo /usr/sbin/setenforce permissive 2024-03-13T21:23:24.040 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-03-13T21:23:24.069 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-03-13T21:23:24.096 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-13T21:23:24.107 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-13T21:23:24.113 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-13T21:23:24.243 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-13T21:23:24.258 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-13T21:23:24.259 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi023.front.sepia.ceph.com,smithi064.front.sepia.ceph.com,smithi077.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-13T21:29:21.988 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi023.front.sepia.ceph.com'), Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi077.front.sepia.ceph.com')] 2024-03-13T21:29:21.989 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-03-13T21:29:21.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:29:22.067 DEBUG:teuthology.orchestra.run.smithi023:> true 2024-03-13T21:29:22.137 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-03-13T21:29:22.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-13T21:29:22.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:29:22.212 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-03-13T21:29:22.287 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-03-13T21:29:22.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-03-13T21:29:22.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T21:29:22.363 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-03-13T21:29:22.434 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-03-13T21:29:22.434 INFO:teuthology.run_tasks:Running task clock... 2024-03-13T21:29:22.445 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-13T21:29:22.445 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-13T21:29:22.446 DEBUG:teuthology.orchestra.run.smithi023:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T21:29:22.448 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-13T21:29:22.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T21:29:22.450 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-13T21:29:22.451 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T21:29:22.483 INFO:teuthology.orchestra.run.smithi023.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-13T21:29:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-13T21:29:22.498 INFO:teuthology.orchestra.run.smithi023.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-13T21:29:22.500 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-13T21:29:22.507 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-13T21:29:22.521 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-13T21:29:22.562 INFO:teuthology.orchestra.run.smithi023.stderr:sudo: ntpd: command not found 2024-03-13T21:29:22.570 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-03-13T21:29:22.574 INFO:teuthology.orchestra.run.smithi023.stdout:506 Cannot talk to daemon 2024-03-13T21:29:22.582 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-03-13T21:29:22.584 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-03-13T21:29:22.588 INFO:teuthology.orchestra.run.smithi023.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-13T21:29:22.595 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-03-13T21:29:22.597 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-13T21:29:22.603 INFO:teuthology.orchestra.run.smithi023.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-13T21:29:22.609 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-13T21:29:22.611 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-13T21:29:22.621 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-13T21:29:22.668 INFO:teuthology.orchestra.run.smithi023.stderr:bash: ntpq: command not found 2024-03-13T21:29:22.671 INFO:teuthology.orchestra.run.smithi023.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T21:29:22.671 INFO:teuthology.orchestra.run.smithi023.stdout:=============================================================================== 2024-03-13T21:29:22.672 INFO:teuthology.orchestra.run.smithi023.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.672 INFO:teuthology.orchestra.run.smithi023.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.672 INFO:teuthology.orchestra.run.smithi023.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.672 INFO:teuthology.orchestra.run.smithi023.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.680 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2024-03-13T21:29:22.684 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T21:29:22.684 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-03-13T21:29:22.684 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.684 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.684 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.684 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.690 INFO:teuthology.orchestra.run.smithi077.stderr:bash: ntpq: command not found 2024-03-13T21:29:22.694 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T21:29:22.694 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-03-13T21:29:22.694 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.694 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.694 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.694 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T21:29:22.695 INFO:teuthology.run_tasks:Running task install... 2024-03-13T21:29:22.705 DEBUG:teuthology.task.install:project ceph 2024-03-13T21:29:22.706 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3'}} 2024-03-13T21:29:22.706 DEBUG:teuthology.task.install:config {'branch': 'octopus', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3'} 2024-03-13T21:29:22.706 INFO:teuthology.task.install:Using flavor: default 2024-03-13T21:29:22.712 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-13T21:29:22.712 INFO:teuthology.task.install:extra packages: ['librados2'] 2024-03-13T21:29:22.713 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'octopus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3', 'tag': None, 'wait_for_package': False} 2024-03-13T21:29:22.713 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T21:29:22.713 INFO:teuthology.packaging:ref: None 2024-03-13T21:29:22.713 INFO:teuthology.packaging:tag: None 2024-03-13T21:29:22.713 INFO:teuthology.packaging:branch: octopus 2024-03-13T21:29:22.713 INFO:teuthology.packaging:sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 2024-03-13T21:29:22.714 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2024-03-13T21:29:22.716 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'octopus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3', 'tag': None, 'wait_for_package': False} 2024-03-13T21:29:22.716 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T21:29:22.716 INFO:teuthology.packaging:ref: None 2024-03-13T21:29:22.716 INFO:teuthology.packaging:tag: None 2024-03-13T21:29:22.716 INFO:teuthology.packaging:branch: octopus 2024-03-13T21:29:22.716 INFO:teuthology.packaging:sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 2024-03-13T21:29:22.716 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2024-03-13T21:29:22.717 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'octopus', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3', 'tag': None, 'wait_for_package': False} 2024-03-13T21:29:22.717 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T21:29:22.718 INFO:teuthology.packaging:ref: None 2024-03-13T21:29:22.718 INFO:teuthology.packaging:tag: None 2024-03-13T21:29:22.718 INFO:teuthology.packaging:branch: octopus 2024-03-13T21:29:22.718 INFO:teuthology.packaging:sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 2024-03-13T21:29:22.718 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2024-03-13T21:29:22.940 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/ 2024-03-13T21:29:22.941 INFO:teuthology.task.install.rpm:Package version is 15.2.17-0 2024-03-13T21:29:22.944 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/ 2024-03-13T21:29:22.945 INFO:teuthology.task.install.rpm:Package version is 15.2.17-0 2024-03-13T21:29:22.987 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-13T21:29:22.988 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-13T21:29:22.988 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-13T21:29:23.006 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-13T21:29:23.006 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:29:23.006 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-13T21:29:23.017 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2024-03-13T21:29:23.017 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T21:29:23.017 INFO:teuthology.packaging:ref: None 2024-03-13T21:29:23.017 INFO:teuthology.packaging:tag: None 2024-03-13T21:29:23.017 INFO:teuthology.packaging:branch: octopus 2024-03-13T21:29:23.017 INFO:teuthology.packaging:sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 2024-03-13T21:29:23.018 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/octopus/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-13T21:29:23.032 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/ 2024-03-13T21:29:23.032 INFO:teuthology.task.install.rpm:Package version is 15.2.17-0 2024-03-13T21:29:23.045 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2024-03-13T21:29:23.045 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T21:29:23.045 INFO:teuthology.packaging:ref: None 2024-03-13T21:29:23.045 INFO:teuthology.packaging:tag: None 2024-03-13T21:29:23.045 INFO:teuthology.packaging:branch: octopus 2024-03-13T21:29:23.045 INFO:teuthology.packaging:sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 2024-03-13T21:29:23.045 DEBUG:teuthology.orchestra.run.smithi023:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/octopus/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-13T21:29:23.071 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/octopus/8a82819d84cf884bd39c17e3236e0632ac146dc4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-13T21:29:23.071 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-03-13T21:29:23.071 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-13T21:29:23.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-13T21:29:23.099 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2024-03-13T21:29:23.099 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T21:29:23.099 INFO:teuthology.packaging:ref: None 2024-03-13T21:29:23.099 INFO:teuthology.packaging:tag: None 2024-03-13T21:29:23.099 INFO:teuthology.packaging:branch: octopus 2024-03-13T21:29:23.099 INFO:teuthology.packaging:sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 2024-03-13T21:29:23.099 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/octopus/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-13T21:29:23.116 DEBUG:teuthology.orchestra.run.smithi023:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-13T21:29:23.167 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-13T21:29:23.168 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-13T21:29:23.196 DEBUG:teuthology.orchestra.run.smithi023:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-13T21:29:23.235 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-03-13T21:29:23.236 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-03-13T21:29:23.243 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-13T21:29:23.264 INFO:teuthology.orchestra.run.smithi023.stdout:check_obsoletes = 1 2024-03-13T21:29:23.266 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum clean all 2024-03-13T21:29:23.308 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2024-03-13T21:29:23.310 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2024-03-13T21:29:23.509 INFO:teuthology.orchestra.run.smithi064.stdout:76 files removed 2024-03-13T21:29:23.537 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw 2024-03-13T21:29:23.554 INFO:teuthology.orchestra.run.smithi023.stdout:76 files removed 2024-03-13T21:29:23.576 INFO:teuthology.orchestra.run.smithi077.stdout:76 files removed 2024-03-13T21:29:23.584 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install ceph-radosgw 2024-03-13T21:29:23.603 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw 2024-03-13T21:29:24.049 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 799 kB/s | 68 kB 00:00 2024-03-13T21:29:24.088 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 815 kB/s | 68 kB 00:00 2024-03-13T21:29:24.092 INFO:teuthology.orchestra.run.smithi023.stdout:ceph packages for x86_64 845 kB/s | 68 kB 00:00 2024-03-13T21:29:24.199 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 136 kB/s | 15 kB 00:00 2024-03-13T21:29:24.209 INFO:teuthology.orchestra.run.smithi023.stdout:ceph noarch packages 194 kB/s | 15 kB 00:00 2024-03-13T21:29:24.222 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 155 kB/s | 15 kB 00:00 2024-03-13T21:29:24.292 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 28 kB/s | 1.7 kB 00:00 2024-03-13T21:29:24.301 INFO:teuthology.orchestra.run.smithi023.stdout:ceph source packages 28 kB/s | 1.7 kB 00:00 2024-03-13T21:29:24.322 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 24 kB/s | 1.7 kB 00:00 2024-03-13T21:29:25.150 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - AppStream 33 MB/s | 28 MB 00:00 2024-03-13T21:29:25.310 INFO:teuthology.orchestra.run.smithi023.stdout:CentOS Stream 8 - AppStream 28 MB/s | 28 MB 00:00 2024-03-13T21:29:25.349 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 8 - AppStream 28 MB/s | 28 MB 00:00 2024-03-13T21:29:30.982 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 8 - BaseOS 14 MB/s | 10 MB 00:00 2024-03-13T21:29:31.226 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - BaseOS 12 MB/s | 10 MB 00:00 2024-03-13T21:29:31.302 INFO:teuthology.orchestra.run.smithi023.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 10 MB 00:00 2024-03-13T21:29:32.982 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 8 - Extras 43 kB/s | 18 kB 00:00 2024-03-13T21:29:33.341 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 8 - Extras common packages 28 kB/s | 7.2 kB 00:00 2024-03-13T21:29:33.378 INFO:teuthology.orchestra.run.smithi023.stdout:CentOS Stream 8 - Extras 47 kB/s | 18 kB 00:00 2024-03-13T21:29:33.443 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras 33 kB/s | 18 kB 00:00 2024-03-13T21:29:33.754 INFO:teuthology.orchestra.run.smithi023.stdout:CentOS Stream 8 - Extras common packages 28 kB/s | 7.2 kB 00:00 2024-03-13T21:29:33.827 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - Extras common packages 27 kB/s | 7.2 kB 00:00 2024-03-13T21:29:34.434 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 8 - PowerTools 4.9 MB/s | 4.9 MB 00:01 2024-03-13T21:29:34.978 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 8 - PowerTools 4.7 MB/s | 4.9 MB 00:01 2024-03-13T21:29:35.388 INFO:teuthology.orchestra.run.smithi023.stdout:CentOS Stream 8 - PowerTools 3.2 MB/s | 4.9 MB 00:01 2024-03-13T21:29:35.586 INFO:teuthology.orchestra.run.smithi077.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-03-13T21:29:36.155 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-03-13T21:29:36.238 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 30 MB/s | 16 MB 00:00 2024-03-13T21:29:36.523 INFO:teuthology.orchestra.run.smithi023.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-03-13T21:29:39.532 INFO:teuthology.orchestra.run.smithi023.stdout:Extra Packages for Enterprise Linux 5.6 MB/s | 16 MB 00:02 2024-03-13T21:29:39.962 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 478 kB/s | 24 kB 00:00 2024-03-13T21:29:43.527 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:29:43.529 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:29:43.529 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-03-13T21:29:43.529 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:15.2.17-0.el8 ceph 9.9 M 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:15.2.17-0.el8 ceph 3.5 M 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:15.2.17-0.el8 ceph 1.9 M 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:15.2.17-0.el8 ceph 5.6 M 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:15.2.17-0.el8 ceph 21 M 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:15.2.17-0.el8 ceph 25 k 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-03-13T21:29:43.530 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:15.2.17-0.el8 ceph 508 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:15.2.17-0.el8 ceph 380 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:15.2.17-0.el8 ceph 3.2 M 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:15.2.17-0.el8 ceph 40 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:15.2.17-0.el8 ceph 77 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:15.2.17-0.el8 ceph 164 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:15.2.17-0.el8 ceph 364 k 2024-03-13T21:29:43.531 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:15.2.17-0.el8 ceph 349 k 2024-03-13T21:29:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:15.2.17-0.el8 ceph 113 k 2024-03-13T21:29:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:29:43.532 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:29:43.532 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:29:43.532 INFO:teuthology.orchestra.run.smithi077.stdout:Install 22 Packages 2024-03-13T21:29:43.532 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 2 Packages 2024-03-13T21:29:43.532 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:29:43.532 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 49 M 2024-03-13T21:29:43.533 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:29:43.541 INFO:teuthology.orchestra.run.smithi023.stdout:lab-extras 395 kB/s | 24 kB 00:00 2024-03-13T21:29:44.188 INFO:teuthology.orchestra.run.smithi077.stdout:(1/24): ceph-base-15.2.17-0.el8.x86_64.rpm 37 MB/s | 5.6 MB 00:00 2024-03-13T21:29:44.213 INFO:teuthology.orchestra.run.smithi077.stdout:(2/24): ceph-selinux-15.2.17-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2024-03-13T21:29:44.247 INFO:teuthology.orchestra.run.smithi077.stdout:(3/24): libcephfs2-15.2.17-0.el8.x86_64.rpm 15 MB/s | 508 kB 00:00 2024-03-13T21:29:44.333 INFO:teuthology.orchestra.run.smithi077.stdout:(4/24): libradosstriper1-15.2.17-0.el8.x86_64.r 4.3 MB/s | 380 kB 00:00 2024-03-13T21:29:44.465 INFO:teuthology.orchestra.run.smithi077.stdout:(5/24): librgw2-15.2.17-0.el8.x86_64.rpm 24 MB/s | 3.2 MB 00:00 2024-03-13T21:29:44.490 INFO:teuthology.orchestra.run.smithi077.stdout:(6/24): python3-ceph-argparse-15.2.17-0.el8.x86 1.6 MB/s | 40 kB 00:00 2024-03-13T21:29:44.507 INFO:teuthology.orchestra.run.smithi077.stdout:(7/24): python3-ceph-common-15.2.17-0.el8.x86_6 4.5 MB/s | 77 kB 00:00 2024-03-13T21:29:44.532 INFO:teuthology.orchestra.run.smithi077.stdout:(8/24): python3-cephfs-15.2.17-0.el8.x86_64.rpm 6.4 MB/s | 164 kB 00:00 2024-03-13T21:29:44.566 INFO:teuthology.orchestra.run.smithi077.stdout:(9/24): python3-rados-15.2.17-0.el8.x86_64.rpm 11 MB/s | 364 kB 00:00 2024-03-13T21:29:44.599 INFO:teuthology.orchestra.run.smithi077.stdout:(10/24): python3-rbd-15.2.17-0.el8.x86_64.rpm 10 MB/s | 349 kB 00:00 2024-03-13T21:29:44.625 INFO:teuthology.orchestra.run.smithi077.stdout:(11/24): python3-rgw-15.2.17-0.el8.x86_64.rpm 4.4 MB/s | 113 kB 00:00 2024-03-13T21:29:44.775 INFO:teuthology.orchestra.run.smithi077.stdout:(12/24): ceph-radosgw-15.2.17-0.el8.x86_64.rpm 14 MB/s | 9.9 MB 00:00 2024-03-13T21:29:45.051 INFO:teuthology.orchestra.run.smithi077.stdout:(13/24): ceph-common-15.2.17-0.el8.x86_64.rpm 21 MB/s | 21 MB 00:01 2024-03-13T21:29:45.092 INFO:teuthology.orchestra.run.smithi077.stdout:(14/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 820 kB/s | 259 kB 00:00 2024-03-13T21:29:45.126 INFO:teuthology.orchestra.run.smithi077.stdout:(15/24): librdkafka-1.6.1-1.el8.x86_64.rpm 1.3 MB/s | 677 kB 00:00 2024-03-13T21:29:45.143 INFO:teuthology.orchestra.run.smithi077.stdout:(16/24): python3-prettytable-0.7.2-14.el8.noarc 477 kB/s | 44 kB 00:00 2024-03-13T21:29:45.227 INFO:teuthology.orchestra.run.smithi077.stdout:(17/24): fmt-6.2.1-1.el8.x86_64.rpm 1.1 MB/s | 116 kB 00:00 2024-03-13T21:29:45.252 INFO:teuthology.orchestra.run.smithi077.stdout:(18/24): librabbitmq-0.9.0-5.el8.x86_64.rpm 295 kB/s | 47 kB 00:00 2024-03-13T21:29:45.277 INFO:teuthology.orchestra.run.smithi077.stdout:(19/24): leveldb-1.22-1.el8.x86_64.rpm 3.5 MB/s | 181 kB 00:00 2024-03-13T21:29:45.328 INFO:teuthology.orchestra.run.smithi077.stdout:(20/24): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2024-03-13T21:29:45.353 INFO:teuthology.orchestra.run.smithi077.stdout:(21/24): liboath-2.6.2-3.el8.x86_64.rpm 586 kB/s | 59 kB 00:00 2024-03-13T21:29:45.378 INFO:teuthology.orchestra.run.smithi077.stdout:(22/24): libunwind-1.3.1-3.el8.x86_64.rpm 746 kB/s | 75 kB 00:00 2024-03-13T21:29:45.428 INFO:teuthology.orchestra.run.smithi077.stdout:(23/24): librbd1-15.2.17-0.el8.x86_64.rpm 26 MB/s | 1.9 MB 00:00 2024-03-13T21:29:45.487 INFO:teuthology.orchestra.run.smithi077.stdout:(24/24): librados2-15.2.17-0.el8.x86_64.rpm 22 MB/s | 3.5 MB 00:00 2024-03-13T21:29:45.489 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:29:45.489 INFO:teuthology.orchestra.run.smithi077.stdout:Total 25 MB/s | 49 MB 00:01 2024-03-13T21:29:45.531 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:29:45.576 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:29:45.577 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:29:45.899 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:29:45.902 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:29:47.033 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:29:47.261 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2024-03-13T21:29:47.455 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2024-03-13T21:29:47.518 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:29:47.520 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout: Package Arch Version Repository Size 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-radosgw x86_64 2:15.2.17-0.el8 ceph 9.9 M 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout:Upgrading: 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout: librados2 x86_64 2:15.2.17-0.el8 ceph 3.5 M 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout: librbd1 x86_64 2:15.2.17-0.el8 ceph 1.9 M 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout:Installing dependencies: 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-base x86_64 2:15.2.17-0.el8 ceph 5.6 M 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-common x86_64 2:15.2.17-0.el8 ceph 21 M 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-selinux x86_64 2:15.2.17-0.el8 ceph 25 k 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2024-03-13T21:29:47.521 INFO:teuthology.orchestra.run.smithi023.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: libcephfs2 x86_64 2:15.2.17-0.el8 ceph 508 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: libradosstriper1 x86_64 2:15.2.17-0.el8 ceph 380 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: librgw2 x86_64 2:15.2.17-0.el8 ceph 3.2 M 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: python3-ceph-argparse x86_64 2:15.2.17-0.el8 ceph 40 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: python3-ceph-common x86_64 2:15.2.17-0.el8 ceph 77 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cephfs x86_64 2:15.2.17-0.el8 ceph 164 k 2024-03-13T21:29:47.522 INFO:teuthology.orchestra.run.smithi023.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-03-13T21:29:47.523 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rados x86_64 2:15.2.17-0.el8 ceph 364 k 2024-03-13T21:29:47.523 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rbd x86_64 2:15.2.17-0.el8 ceph 349 k 2024-03-13T21:29:47.523 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rgw x86_64 2:15.2.17-0.el8 ceph 113 k 2024-03-13T21:29:47.523 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:29:47.523 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:29:47.523 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:29:47.523 INFO:teuthology.orchestra.run.smithi023.stdout:Install 22 Packages 2024-03-13T21:29:47.523 INFO:teuthology.orchestra.run.smithi023.stdout:Upgrade 2 Packages 2024-03-13T21:29:47.523 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:29:47.523 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 49 M 2024-03-13T21:29:47.524 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:29:47.642 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2024-03-13T21:29:47.934 INFO:teuthology.orchestra.run.smithi023.stdout:(1/24): ceph-base-15.2.17-0.el8.x86_64.rpm 38 MB/s | 5.6 MB 00:00 2024-03-13T21:29:47.959 INFO:teuthology.orchestra.run.smithi023.stdout:(2/24): ceph-selinux-15.2.17-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2024-03-13T21:29:47.993 INFO:teuthology.orchestra.run.smithi023.stdout:(3/24): libcephfs2-15.2.17-0.el8.x86_64.rpm 15 MB/s | 508 kB 00:00 2024-03-13T21:29:48.005 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2024-03-13T21:29:48.026 INFO:teuthology.orchestra.run.smithi023.stdout:(4/24): libradosstriper1-15.2.17-0.el8.x86_64.r 11 MB/s | 380 kB 00:00 2024-03-13T21:29:48.118 INFO:teuthology.orchestra.run.smithi023.stdout:(5/24): librgw2-15.2.17-0.el8.x86_64.rpm 35 MB/s | 3.2 MB 00:00 2024-03-13T21:29:48.144 INFO:teuthology.orchestra.run.smithi023.stdout:(6/24): python3-ceph-argparse-15.2.17-0.el8.x86 1.6 MB/s | 40 kB 00:00 2024-03-13T21:29:48.169 INFO:teuthology.orchestra.run.smithi023.stdout:(7/24): python3-ceph-common-15.2.17-0.el8.x86_6 3.0 MB/s | 77 kB 00:00 2024-03-13T21:29:48.194 INFO:teuthology.orchestra.run.smithi023.stdout:(8/24): python3-cephfs-15.2.17-0.el8.x86_64.rpm 6.4 MB/s | 164 kB 00:00 2024-03-13T21:29:48.217 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:15.2.17-0.el8.x86_64 4/26 2024-03-13T21:29:48.219 INFO:teuthology.orchestra.run.smithi023.stdout:(9/24): python3-rados-15.2.17-0.el8.x86_64.rpm 14 MB/s | 364 kB 00:00 2024-03-13T21:29:48.314 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:15.2.17-0.el8.x86_64 4/26 2024-03-13T21:29:48.328 INFO:teuthology.orchestra.run.smithi023.stdout:(10/24): ceph-radosgw-15.2.17-0.el8.x86_64.rpm 18 MB/s | 9.9 MB 00:00 2024-03-13T21:29:48.353 INFO:teuthology.orchestra.run.smithi023.stdout:(11/24): python3-rbd-15.2.17-0.el8.x86_64.rpm 2.6 MB/s | 349 kB 00:00 2024-03-13T21:29:48.370 INFO:teuthology.orchestra.run.smithi023.stdout:(12/24): python3-rgw-15.2.17-0.el8.x86_64.rpm 2.6 MB/s | 113 kB 00:00 2024-03-13T21:29:48.565 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:15.2.17-0.el8.x86_64 5/26 2024-03-13T21:29:48.595 INFO:teuthology.orchestra.run.smithi023.stdout:(13/24): ceph-common-15.2.17-0.el8.x86_64.rpm 26 MB/s | 21 MB 00:00 2024-03-13T21:29:48.637 INFO:teuthology.orchestra.run.smithi023.stdout:(14/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 972 kB/s | 259 kB 00:00 2024-03-13T21:29:48.671 INFO:teuthology.orchestra.run.smithi023.stdout:(15/24): librdkafka-1.6.1-1.el8.x86_64.rpm 2.1 MB/s | 677 kB 00:00 2024-03-13T21:29:48.697 INFO:teuthology.orchestra.run.smithi023.stdout:(16/24): python3-prettytable-0.7.2-14.el8.noarc 437 kB/s | 44 kB 00:00 2024-03-13T21:29:48.759 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2024-03-13T21:29:48.789 INFO:teuthology.orchestra.run.smithi023.stdout:(17/24): librabbitmq-0.9.0-5.el8.x86_64.rpm 311 kB/s | 47 kB 00:00 2024-03-13T21:29:48.985 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2024-03-13T21:29:49.131 INFO:teuthology.orchestra.run.smithi023.stdout:(18/24): fmt-6.2.1-1.el8.x86_64.rpm 253 kB/s | 116 kB 00:00 2024-03-13T21:29:49.214 INFO:teuthology.orchestra.run.smithi023.stdout:(19/24): leveldb-1.22-1.el8.x86_64.rpm 426 kB/s | 181 kB 00:00 2024-03-13T21:29:49.258 INFO:teuthology.orchestra.run.smithi023.stdout:(20/24): gperftools-libs-2.7-9.el8.x86_64.rpm 546 kB/s | 306 kB 00:00 2024-03-13T21:29:49.267 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:15.2.17-0.el8.x86_64 7/26 2024-03-13T21:29:49.334 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:15.2.17-0.el8.x86_64 7/26 2024-03-13T21:29:49.340 INFO:teuthology.orchestra.run.smithi023.stdout:(21/24): librados2-15.2.17-0.el8.x86_64.rpm 43 MB/s | 3.5 MB 00:00 2024-03-13T21:29:49.415 INFO:teuthology.orchestra.run.smithi023.stdout:(22/24): librbd1-15.2.17-0.el8.x86_64.rpm 26 MB/s | 1.9 MB 00:00 2024-03-13T21:29:49.482 INFO:teuthology.orchestra.run.smithi023.stdout:(23/24): liboath-2.6.2-3.el8.x86_64.rpm 167 kB/s | 59 kB 00:00 2024-03-13T21:29:49.607 INFO:teuthology.orchestra.run.smithi023.stdout:(24/24): libunwind-1.3.1-3.el8.x86_64.rpm 191 kB/s | 75 kB 00:00 2024-03-13T21:29:49.610 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:29:49.611 INFO:teuthology.orchestra.run.smithi023.stdout:Total 23 MB/s | 49 MB 00:02 2024-03-13T21:29:49.618 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/26 2024-03-13T21:29:49.655 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:29:49.706 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:29:49.706 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:29:50.027 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:29:50.030 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:29:50.085 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/26 2024-03-13T21:29:50.275 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:15.2.17-0.el8.x86_64 10/26 2024-03-13T21:29:50.351 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:15.2.17-0.el8.x86_64 10/26 2024-03-13T21:29:50.563 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2024-03-13T21:29:50.717 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 12/26 2024-03-13T21:29:50.946 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:15.2.17-0.el8.x86_64 13/26 2024-03-13T21:29:51.183 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:15.2.17-0.el8.x86_64 14/26 2024-03-13T21:29:51.188 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:29:51.408 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:15.2.17-0.el8.x86_64 15/26 2024-03-13T21:29:51.420 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2024-03-13T21:29:51.567 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2024-03-13T21:29:51.633 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2024-03-13T21:29:51.665 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2024-03-13T21:29:51.803 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2024-03-13T21:29:51.894 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2024-03-13T21:29:52.109 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/26 2024-03-13T21:29:52.187 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2024-03-13T21:29:52.311 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2024-03-13T21:29:52.386 INFO:teuthology.orchestra.run.smithi023.stdout: Upgrading : librados2-2:15.2.17-0.el8.x86_64 4/26 2024-03-13T21:29:52.443 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 16 MB 00:16 2024-03-13T21:29:52.501 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:15.2.17-0.el8.x86_64 20/26 2024-03-13T21:29:52.515 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librados2-2:15.2.17-0.el8.x86_64 4/26 2024-03-13T21:29:52.762 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-rados-2:15.2.17-0.el8.x86_64 5/26 2024-03-13T21:29:53.078 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2024-03-13T21:29:53.313 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2024-03-13T21:29:53.519 INFO:teuthology.orchestra.run.smithi023.stdout: Upgrading : librbd1-2:15.2.17-0.el8.x86_64 7/26 2024-03-13T21:29:53.614 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librbd1-2:15.2.17-0.el8.x86_64 7/26 2024-03-13T21:29:53.833 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/26 2024-03-13T21:29:54.333 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/26 2024-03-13T21:29:54.503 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : librgw2-2:15.2.17-0.el8.x86_64 10/26 2024-03-13T21:29:54.562 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librgw2-2:15.2.17-0.el8.x86_64 10/26 2024-03-13T21:29:54.733 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2024-03-13T21:29:54.749 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2024-03-13T21:29:54.913 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 12/26 2024-03-13T21:29:54.975 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:15.2.17-0.el8.x86_64 21/26 2024-03-13T21:29:55.074 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cephfs-2:15.2.17-0.el8.x86_64 13/26 2024-03-13T21:29:55.296 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-rgw-2:15.2.17-0.el8.x86_64 14/26 2024-03-13T21:29:55.505 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-rbd-2:15.2.17-0.el8.x86_64 15/26 2024-03-13T21:29:55.636 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2024-03-13T21:29:55.802 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2024-03-13T21:29:55.876 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2024-03-13T21:29:55.910 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:15.2.17-0.el8.x86_64 22/26 2024-03-13T21:29:56.001 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:15.2.17-0.el8.x86_64 22/26 2024-03-13T21:29:56.091 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2024-03-13T21:29:56.174 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2024-03-13T21:29:56.262 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/26 2024-03-13T21:29:56.455 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 513 kB/s | 24 kB 00:00 2024-03-13T21:29:56.467 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2024-03-13T21:29:56.637 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-ceph-common-2:15.2.17-0.el8.x86_64 20/26 2024-03-13T21:29:59.052 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2024-03-13T21:29:59.411 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-common-2:15.2.17-0.el8.x86_64 21/26 2024-03-13T21:30:00.109 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2024-03-13T21:30:00.256 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:15.2.17-0.el8 ceph 9.9 M 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:15.2.17-0.el8 ceph 3.5 M 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:15.2.17-0.el8 ceph 1.9 M 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:15.2.17-0.el8 ceph 5.6 M 2024-03-13T21:30:00.259 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:15.2.17-0.el8 ceph 21 M 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:15.2.17-0.el8 ceph 25 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:15.2.17-0.el8 ceph 508 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:15.2.17-0.el8 ceph 380 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:15.2.17-0.el8 ceph 3.2 M 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:15.2.17-0.el8 ceph 40 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:15.2.17-0.el8 ceph 77 k 2024-03-13T21:30:00.260 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:15.2.17-0.el8 ceph 164 k 2024-03-13T21:30:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-03-13T21:30:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:15.2.17-0.el8 ceph 364 k 2024-03-13T21:30:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:15.2.17-0.el8 ceph 349 k 2024-03-13T21:30:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:15.2.17-0.el8 ceph 113 k 2024-03-13T21:30:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:00.261 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:30:00.261 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:30:00.261 INFO:teuthology.orchestra.run.smithi064.stdout:Install 22 Packages 2024-03-13T21:30:00.261 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 2 Packages 2024-03-13T21:30:00.261 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:00.262 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 49 M 2024-03-13T21:30:00.262 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:30:00.326 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-base-2:15.2.17-0.el8.x86_64 22/26 2024-03-13T21:30:00.437 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-base-2:15.2.17-0.el8.x86_64 22/26 2024-03-13T21:30:00.593 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2024-03-13T21:30:00.681 INFO:teuthology.orchestra.run.smithi064.stdout:(1/24): ceph-base-15.2.17-0.el8.x86_64.rpm 33 MB/s | 5.6 MB 00:00 2024-03-13T21:30:00.707 INFO:teuthology.orchestra.run.smithi064.stdout:(2/24): ceph-selinux-15.2.17-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2024-03-13T21:30:00.740 INFO:teuthology.orchestra.run.smithi064.stdout:(3/24): libcephfs2-15.2.17-0.el8.x86_64.rpm 15 MB/s | 508 kB 00:00 2024-03-13T21:30:00.774 INFO:teuthology.orchestra.run.smithi064.stdout:(4/24): libradosstriper1-15.2.17-0.el8.x86_64.r 11 MB/s | 380 kB 00:00 2024-03-13T21:30:00.857 INFO:teuthology.orchestra.run.smithi064.stdout:(5/24): librgw2-15.2.17-0.el8.x86_64.rpm 39 MB/s | 3.2 MB 00:00 2024-03-13T21:30:00.883 INFO:teuthology.orchestra.run.smithi064.stdout:(6/24): python3-ceph-argparse-15.2.17-0.el8.x86 1.6 MB/s | 40 kB 00:00 2024-03-13T21:30:00.933 INFO:teuthology.orchestra.run.smithi064.stdout:(7/24): python3-ceph-common-15.2.17-0.el8.x86_6 1.5 MB/s | 77 kB 00:00 2024-03-13T21:30:00.958 INFO:teuthology.orchestra.run.smithi064.stdout:(8/24): python3-cephfs-15.2.17-0.el8.x86_64.rpm 6.5 MB/s | 164 kB 00:00 2024-03-13T21:30:00.992 INFO:teuthology.orchestra.run.smithi064.stdout:(9/24): python3-rados-15.2.17-0.el8.x86_64.rpm 11 MB/s | 364 kB 00:00 2024-03-13T21:30:01.026 INFO:teuthology.orchestra.run.smithi064.stdout:(10/24): python3-rbd-15.2.17-0.el8.x86_64.rpm 10 MB/s | 349 kB 00:00 2024-03-13T21:30:01.059 INFO:teuthology.orchestra.run.smithi064.stdout:(11/24): python3-rgw-15.2.17-0.el8.x86_64.rpm 3.3 MB/s | 113 kB 00:00 2024-03-13T21:30:01.168 INFO:teuthology.orchestra.run.smithi064.stdout:(12/24): ceph-radosgw-15.2.17-0.el8.x86_64.rpm 15 MB/s | 9.9 MB 00:00 2024-03-13T21:30:01.419 INFO:teuthology.orchestra.run.smithi064.stdout:(13/24): ceph-common-15.2.17-0.el8.x86_64.rpm 23 MB/s | 21 MB 00:00 2024-03-13T21:30:01.460 INFO:teuthology.orchestra.run.smithi064.stdout:(14/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 890 kB/s | 259 kB 00:00 2024-03-13T21:30:01.527 INFO:teuthology.orchestra.run.smithi064.stdout:(15/24): librdkafka-1.6.1-1.el8.x86_64.rpm 1.4 MB/s | 677 kB 00:00 2024-03-13T21:30:01.553 INFO:teuthology.orchestra.run.smithi064.stdout:(16/24): python3-prettytable-0.7.2-14.el8.noarc 328 kB/s | 44 kB 00:00 2024-03-13T21:30:01.645 INFO:teuthology.orchestra.run.smithi064.stdout:(17/24): librabbitmq-0.9.0-5.el8.x86_64.rpm 256 kB/s | 47 kB 00:00 2024-03-13T21:30:02.162 INFO:teuthology.orchestra.run.smithi064.stdout:(18/24): fmt-6.2.1-1.el8.x86_64.rpm 183 kB/s | 116 kB 00:00 2024-03-13T21:30:02.279 INFO:teuthology.orchestra.run.smithi064.stdout:(19/24): leveldb-1.22-1.el8.x86_64.rpm 285 kB/s | 181 kB 00:00 2024-03-13T21:30:02.304 INFO:teuthology.orchestra.run.smithi064.stdout:(20/24): gperftools-libs-2.7-9.el8.x86_64.rpm 408 kB/s | 306 kB 00:00 2024-03-13T21:30:02.329 INFO:teuthology.orchestra.run.smithi064.stdout:(21/24): liboath-2.6.2-3.el8.x86_64.rpm 352 kB/s | 59 kB 00:00 2024-03-13T21:30:02.396 INFO:teuthology.orchestra.run.smithi064.stdout:(22/24): librbd1-15.2.17-0.el8.x86_64.rpm 29 MB/s | 1.9 MB 00:00 2024-03-13T21:30:02.421 INFO:teuthology.orchestra.run.smithi064.stdout:(23/24): libunwind-1.3.1-3.el8.x86_64.rpm 527 kB/s | 75 kB 00:00 2024-03-13T21:30:02.480 INFO:teuthology.orchestra.run.smithi064.stdout:(24/24): librados2-15.2.17-0.el8.x86_64.rpm 20 MB/s | 3.5 MB 00:00 2024-03-13T21:30:02.485 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:30:02.485 INFO:teuthology.orchestra.run.smithi064.stdout:Total 22 MB/s | 49 MB 00:02 2024-03-13T21:30:02.528 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:30:02.579 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:30:02.579 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:30:02.918 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:30:02.921 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:30:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:30:04.290 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2024-03-13T21:30:04.521 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/26 2024-03-13T21:30:04.720 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/26 2024-03-13T21:30:05.101 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/26 2024-03-13T21:30:05.303 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:15.2.17-0.el8.x86_64 4/26 2024-03-13T21:30:05.398 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:15.2.17-0.el8.x86_64 4/26 2024-03-13T21:30:05.663 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:15.2.17-0.el8.x86_64 5/26 2024-03-13T21:30:05.820 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2024-03-13T21:30:06.087 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:15.2.17-0.el8.x86_64 6/26 2024-03-13T21:30:06.294 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:15.2.17-0.el8.x86_64 7/26 2024-03-13T21:30:06.347 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:15.2.17-0.el8.x86_64 7/26 2024-03-13T21:30:06.598 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/26 2024-03-13T21:30:07.086 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/26 2024-03-13T21:30:07.228 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:15.2.17-0.el8.x86_64 10/26 2024-03-13T21:30:07.314 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:15.2.17-0.el8.x86_64 10/26 2024-03-13T21:30:07.500 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2024-03-13T21:30:07.680 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 12/26 2024-03-13T21:30:07.983 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:15.2.17-0.el8.x86_64 13/26 2024-03-13T21:30:08.181 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:15.2.17-0.el8.x86_64 14/26 2024-03-13T21:30:08.379 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:15.2.17-0.el8.x86_64 15/26 2024-03-13T21:30:08.578 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2024-03-13T21:30:08.649 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:15.2.17-0.el8.x86_64 16/26 2024-03-13T21:30:08.891 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/26 2024-03-13T21:30:09.088 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/26 2024-03-13T21:30:09.285 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 19/26 2024-03-13T21:30:09.470 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:15.2.17-0.el8.x86_64 20/26 2024-03-13T21:30:11.875 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2024-03-13T21:30:12.086 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:15.2.17-0.el8.x86_64 21/26 2024-03-13T21:30:12.782 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:15.2.17-0.el8.x86_64 21/26 2024-03-13T21:30:13.040 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:15.2.17-0.el8.x86_64 22/26 2024-03-13T21:30:13.144 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:15.2.17-0.el8.x86_64 22/26 2024-03-13T21:30:13.294 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2024-03-13T21:30:15.002 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2024-03-13T21:30:15.003 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2024-03-13T21:30:15.003 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2024-03-13T21:30:15.003 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /dev 2024-03-13T21:30:15.003 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /run 2024-03-13T21:30:15.003 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2024-03-13T21:30:15.003 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2024-03-13T21:30:15.003 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2024-03-13T21:30:15.003 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2024-03-13T21:30:15.003 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /dev 2024-03-13T21:30:15.003 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:16.049 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2024-03-13T21:30:16.126 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2024-03-13T21:30:16.126 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2024-03-13T21:30:16.366 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2024-03-13T21:30:16.366 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2024-03-13T21:30:17.071 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2024-03-13T21:30:17.071 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:15.2.17-0.el8.x86_64 1/26 2024-03-13T21:30:17.071 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:15.2.17-0.el8.x86_64 2/26 2024-03-13T21:30:17.071 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:15.2.17-0.el8.x86_64 3/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:15.2.17-0.el8.x86_64 4/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:15.2.17-0.el8.x86_64 5/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:15.2.17-0.el8.x86_64 6/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:15.2.17-0.el8.x86_64 7/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 8/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:15.2.17-0.el8.x86_64 9/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:15.2.17-0.el8.x86_64 10/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:15.2.17-0.el8.x86_64 11/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:15.2.17-0.el8.x86_64 12/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:15.2.17-0.el8.x86_64 13/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2024-03-13T21:30:17.072 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2024-03-13T21:30:17.073 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/26 2024-03-13T21:30:17.073 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2024-03-13T21:30:17.073 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 19/26 2024-03-13T21:30:17.073 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2024-03-13T21:30:17.073 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2024-03-13T21:30:17.073 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2024-03-13T21:30:17.073 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:15.2.17-0.el8.x86_64 23/26 2024-03-13T21:30:17.073 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2024-03-13T21:30:17.073 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:15.2.17-0.el8.x86_64 25/26 2024-03-13T21:30:18.053 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2024-03-13T21:30:18.053 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:15.2.17-0.el8.x86_64 librbd1-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout: fmt-6.2.1-1.el8.x86_64 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout: leveldb-1.22-1.el8.x86_64 2024-03-13T21:30:18.054 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.055 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.056 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:18.056 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:18.056 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:30:18.378 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-test 2024-03-13T21:30:19.128 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:40 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:30:20.531 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2024-03-13T21:30:20.531 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /sys 2024-03-13T21:30:20.531 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /proc 2024-03-13T21:30:20.531 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /dev 2024-03-13T21:30:20.531 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /run 2024-03-13T21:30:20.531 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /mnt 2024-03-13T21:30:20.531 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /var/tmp 2024-03-13T21:30:20.531 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /home 2024-03-13T21:30:20.532 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /tmp 2024-03-13T21:30:20.532 INFO:teuthology.orchestra.run.smithi023.stdout:skipping the directory /dev 2024-03-13T21:30:20.532 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:20.726 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:30:20.726 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:15.2.17-0.el8 ceph 42 M 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:20.727 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:30:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:Install 5 Packages 2024-03-13T21:30:20.728 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 43 M 2024-03-13T21:30:20.728 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 155 M 2024-03-13T21:30:20.729 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:30:21.267 INFO:teuthology.orchestra.run.smithi077.stdout:(1/5): oniguruma-6.8.2-3.el8.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-03-13T21:30:21.292 INFO:teuthology.orchestra.run.smithi077.stdout:(2/5): jq-1.6-8.el8.x86_64.rpm 1.2 MB/s | 203 kB 00:00 2024-03-13T21:30:21.334 INFO:teuthology.orchestra.run.smithi077.stdout:(3/5): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2024-03-13T21:30:21.359 INFO:teuthology.orchestra.run.smithi077.stdout:(4/5): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-03-13T21:30:21.590 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2024-03-13T21:30:21.654 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2024-03-13T21:30:21.655 INFO:teuthology.orchestra.run.smithi023.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2024-03-13T21:30:21.802 INFO:teuthology.orchestra.run.smithi077.stdout:(5/5): ceph-test-15.2.17-0.el8.x86_64.rpm 63 MB/s | 42 MB 00:00 2024-03-13T21:30:21.804 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:30:21.804 INFO:teuthology.orchestra.run.smithi077.stdout:Total 40 MB/s | 43 MB 00:01 2024-03-13T21:30:21.846 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:30:21.864 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:30:21.865 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:30:21.893 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2024-03-13T21:30:21.893 INFO:teuthology.orchestra.run.smithi023.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2024-03-13T21:30:22.154 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:30:22.157 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:30:22.567 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:30:22.608 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2024-03-13T21:30:22.608 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-base-2:15.2.17-0.el8.x86_64 1/26 2024-03-13T21:30:22.608 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-common-2:15.2.17-0.el8.x86_64 2/26 2024-03-13T21:30:22.608 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-radosgw-2:15.2.17-0.el8.x86_64 3/26 2024-03-13T21:30:22.608 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-selinux-2:15.2.17-0.el8.x86_64 4/26 2024-03-13T21:30:22.608 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libcephfs2-2:15.2.17-0.el8.x86_64 5/26 2024-03-13T21:30:22.608 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libradosstriper1-2:15.2.17-0.el8.x86_64 6/26 2024-03-13T21:30:22.608 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librgw2-2:15.2.17-0.el8.x86_64 7/26 2024-03-13T21:30:22.608 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 8/26 2024-03-13T21:30:22.608 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-ceph-common-2:15.2.17-0.el8.x86_64 9/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cephfs-2:15.2.17-0.el8.x86_64 10/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-rados-2:15.2.17-0.el8.x86_64 11/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-rbd-2:15.2.17-0.el8.x86_64 12/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-rgw-2:15.2.17-0.el8.x86_64 13/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 19/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2024-03-13T21:30:22.609 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2024-03-13T21:30:22.610 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2024-03-13T21:30:22.610 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librados2-2:15.2.17-0.el8.x86_64 23/26 2024-03-13T21:30:22.610 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2024-03-13T21:30:22.610 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librbd1-2:15.2.17-0.el8.x86_64 25/26 2024-03-13T21:30:22.823 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/5 2024-03-13T21:30:23.032 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/5 2024-03-13T21:30:23.196 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/5 2024-03-13T21:30:23.290 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/5 2024-03-13T21:30:23.582 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2024-03-13T21:30:23.582 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:23.582 INFO:teuthology.orchestra.run.smithi023.stdout:Upgraded: 2024-03-13T21:30:23.582 INFO:teuthology.orchestra.run.smithi023.stdout: librados2-2:15.2.17-0.el8.x86_64 librbd1-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.582 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:30:23.582 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-base-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.582 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-common-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-radosgw-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-selinux-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: fmt-6.2.1-1.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: leveldb-1.22-1.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: libcephfs2-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: libradosstriper1-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: librgw2-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: python3-ceph-argparse-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.583 INFO:teuthology.orchestra.run.smithi023.stdout: python3-ceph-common-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.584 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cephfs-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.584 INFO:teuthology.orchestra.run.smithi023.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-13T21:30:23.584 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rados-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.584 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rbd-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.584 INFO:teuthology.orchestra.run.smithi023.stdout: python3-rgw-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:23.584 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:23.584 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:30:23.915 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install ceph-test 2024-03-13T21:30:24.698 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:00:41 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:30:26.397 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:30:26.398 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:30:26.398 INFO:teuthology.orchestra.run.smithi023.stdout: Package Architecture Version Repository Size 2024-03-13T21:30:26.398 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-test x86_64 2:15.2.17-0.el8 ceph 42 M 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout:Installing dependencies: 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout:Install 5 Packages 2024-03-13T21:30:26.399 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:26.400 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 43 M 2024-03-13T21:30:26.400 INFO:teuthology.orchestra.run.smithi023.stdout:Installed size: 155 M 2024-03-13T21:30:26.400 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:30:26.738 INFO:teuthology.orchestra.run.smithi023.stdout:(1/5): jq-1.6-8.el8.x86_64.rpm 1.5 MB/s | 203 kB 00:00 2024-03-13T21:30:26.763 INFO:teuthology.orchestra.run.smithi023.stdout:(2/5): oniguruma-6.8.2-3.el8.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-03-13T21:30:26.797 INFO:teuthology.orchestra.run.smithi023.stdout:(3/5): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2024-03-13T21:30:26.822 INFO:teuthology.orchestra.run.smithi023.stdout:(4/5): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-13T21:30:27.157 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : jq-1.6-8.el8.x86_64 4/5 2024-03-13T21:30:27.256 INFO:teuthology.orchestra.run.smithi023.stdout:(5/5): ceph-test-15.2.17-0.el8.x86_64.rpm 65 MB/s | 42 MB 00:00 2024-03-13T21:30:27.256 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:30:27.257 INFO:teuthology.orchestra.run.smithi023.stdout:Total 50 MB/s | 43 MB 00:00 2024-03-13T21:30:27.298 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:30:27.318 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:30:27.319 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:30:27.411 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:15.2.17-0.el8.x86_64 5/5 2024-03-13T21:30:27.615 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:30:27.618 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:30:27.708 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-test-2:15.2.17-0.el8.x86_64 5/5 2024-03-13T21:30:27.708 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:15.2.17-0.el8.x86_64 1/5 2024-03-13T21:30:27.708 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : jq-1.6-8.el8.x86_64 2/5 2024-03-13T21:30:27.708 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 3/5 2024-03-13T21:30:27.708 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/5 2024-03-13T21:30:28.065 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:30:28.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 5/5 2024-03-13T21:30:28.134 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:28.134 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:30:28.134 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:15.2.17-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-03-13T21:30:28.134 INFO:teuthology.orchestra.run.smithi077.stdout: oniguruma-6.8.2-3.el8.x86_64 socat-1.7.4.1-1.el8.x86_64 2024-03-13T21:30:28.134 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-13T21:30:28.135 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:28.135 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:30:28.282 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/5 2024-03-13T21:30:28.356 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph 2024-03-13T21:30:28.460 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/5 2024-03-13T21:30:28.598 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/5 2024-03-13T21:30:28.677 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/5 2024-03-13T21:30:29.122 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:50 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:30:30.731 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:15.2.17-0.el8 ceph 6.3 k 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:15.2.17-0.el8 ceph 2.0 M 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:15.2.17-0.el8 ceph 1.4 M 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:15.2.17-0.el8 ceph-noarch 193 k 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:15.2.17-0.el8 ceph 4.3 M 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:15.2.17-0.el8 ceph 17 M 2024-03-13T21:30:30.735 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-13T21:30:30.736 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-13T21:30:30.737 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:30.738 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:30:30.738 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:30.738 INFO:teuthology.orchestra.run.smithi077.stdout:Install 32 Packages 2024-03-13T21:30:30.738 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:30.738 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 28 M 2024-03-13T21:30:30.738 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 91 M 2024-03-13T21:30:30.739 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:30:31.285 INFO:teuthology.orchestra.run.smithi077.stdout:(1/32): ceph-15.2.17-0.el8.x86_64.rpm 140 kB/s | 6.3 kB 00:00 2024-03-13T21:30:31.335 INFO:teuthology.orchestra.run.smithi077.stdout:(2/32): ceph-mds-15.2.17-0.el8.x86_64.rpm 21 MB/s | 2.0 MB 00:00 2024-03-13T21:30:31.369 INFO:teuthology.orchestra.run.smithi077.stdout:(3/32): ceph-mgr-15.2.17-0.el8.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-03-13T21:30:31.434 INFO:teuthology.orchestra.run.smithi077.stdout:(4/32): ceph-mgr-modules-core-15.2.17-0.el8.noa 2.9 MB/s | 193 kB 00:00 2024-03-13T21:30:31.511 INFO:teuthology.orchestra.run.smithi077.stdout:(5/32): ceph-mon-15.2.17-0.el8.x86_64.rpm 19 MB/s | 4.3 MB 00:00 2024-03-13T21:30:31.746 INFO:teuthology.orchestra.run.smithi077.stdout:(6/32): ceph-osd-15.2.17-0.el8.x86_64.rpm 41 MB/s | 17 MB 00:00 2024-03-13T21:30:31.770 INFO:teuthology.orchestra.run.smithi077.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 150 kB/s | 39 kB 00:00 2024-03-13T21:30:31.795 INFO:teuthology.orchestra.run.smithi077.stdout:(8/32): python3-mako-1.0.6-14.el8.noarch.rpm 435 kB/s | 157 kB 00:00 2024-03-13T21:30:31.829 INFO:teuthology.orchestra.run.smithi077.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 974 kB/s | 57 kB 00:00 2024-03-13T21:30:31.871 INFO:teuthology.orchestra.run.smithi077.stdout:(10/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 824 kB/s | 103 kB 00:00 2024-03-13T21:30:31.896 INFO:teuthology.orchestra.run.smithi077.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 4.4 MB/s | 457 kB 00:00 2024-03-13T21:30:32.071 INFO:teuthology.orchestra.run.smithi077.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.0 MB/s | 249 kB 00:00 2024-03-13T21:30:32.097 INFO:teuthology.orchestra.run.smithi077.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 545 kB/s | 109 kB 00:00 2024-03-13T21:30:32.197 INFO:teuthology.orchestra.run.smithi077.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 1.9 MB/s | 641 kB 00:00 2024-03-13T21:30:32.464 INFO:teuthology.orchestra.run.smithi077.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 111 kB/s | 44 kB 00:00 2024-03-13T21:30:32.606 INFO:teuthology.orchestra.run.smithi077.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 364 kB/s | 185 kB 00:00 2024-03-13T21:30:32.631 INFO:teuthology.orchestra.run.smithi077.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 398 kB/s | 173 kB 00:00 2024-03-13T21:30:32.723 INFO:teuthology.orchestra.run.smithi077.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2024-03-13T21:30:32.748 INFO:teuthology.orchestra.run.smithi077.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 150 kB/s | 18 kB 00:00 2024-03-13T21:30:32.751 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : jq-1.6-8.el8.x86_64 4/5 2024-03-13T21:30:32.865 INFO:teuthology.orchestra.run.smithi077.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 959 kB/s | 384 kB 00:00 2024-03-13T21:30:32.899 INFO:teuthology.orchestra.run.smithi077.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 278 kB/s | 49 kB 00:00 2024-03-13T21:30:32.924 INFO:teuthology.orchestra.run.smithi077.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 334 kB/s | 59 kB 00:00 2024-03-13T21:30:32.979 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-test-2:15.2.17-0.el8.x86_64 5/5 2024-03-13T21:30:33.049 INFO:teuthology.orchestra.run.smithi077.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.5 MB/s | 283 kB 00:00 2024-03-13T21:30:33.108 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:15.2.17-0.el8.x86_64 23/26 2024-03-13T21:30:33.108 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-03-13T21:30:33.108 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-03-13T21:30:33.108 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2024-03-13T21:30:33.108 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /run 2024-03-13T21:30:33.108 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-03-13T21:30:33.108 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-03-13T21:30:33.108 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-03-13T21:30:33.109 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-03-13T21:30:33.109 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2024-03-13T21:30:33.109 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:33.109 INFO:teuthology.orchestra.run.smithi077.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 85 kB/s | 16 kB 00:00 2024-03-13T21:30:33.133 INFO:teuthology.orchestra.run.smithi077.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 90 kB/s | 19 kB 00:00 2024-03-13T21:30:33.158 INFO:teuthology.orchestra.run.smithi077.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 220 kB/s | 24 kB 00:00 2024-03-13T21:30:33.209 INFO:teuthology.orchestra.run.smithi077.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 291 kB/s | 29 kB 00:00 2024-03-13T21:30:33.234 INFO:teuthology.orchestra.run.smithi077.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 273 kB/s | 27 kB 00:00 2024-03-13T21:30:33.280 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-test-2:15.2.17-0.el8.x86_64 5/5 2024-03-13T21:30:33.281 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-test-2:15.2.17-0.el8.x86_64 1/5 2024-03-13T21:30:33.281 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : jq-1.6-8.el8.x86_64 2/5 2024-03-13T21:30:33.281 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 3/5 2024-03-13T21:30:33.281 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/5 2024-03-13T21:30:33.293 INFO:teuthology.orchestra.run.smithi077.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.9 MB/s | 255 kB 00:00 2024-03-13T21:30:33.385 INFO:teuthology.orchestra.run.smithi077.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 568 kB/s | 85 kB 00:00 2024-03-13T21:30:33.410 INFO:teuthology.orchestra.run.smithi077.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 197 kB/s | 23 kB 00:00 2024-03-13T21:30:33.644 INFO:teuthology.orchestra.run.smithi077.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 578 kB/s | 251 kB 00:00 2024-03-13T21:30:33.645 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:30:33.645 INFO:teuthology.orchestra.run.smithi077.stdout:Total 9.8 MB/s | 28 MB 00:02 2024-03-13T21:30:33.726 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:30:33.764 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:30:33.764 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:30:33.771 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 5/5 2024-03-13T21:30:33.771 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:33.771 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:30:33.771 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-test-2:15.2.17-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-03-13T21:30:33.772 INFO:teuthology.orchestra.run.smithi023.stdout: oniguruma-6.8.2-3.el8.x86_64 socat-1.7.4.1-1.el8.x86_64 2024-03-13T21:30:33.772 INFO:teuthology.orchestra.run.smithi023.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-13T21:30:33.772 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:33.772 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:30:33.952 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install ceph 2024-03-13T21:30:34.064 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:30:34.067 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:30:34.203 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2024-03-13T21:30:34.279 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:15.2.17-0.el8.x86_64 24/26 2024-03-13T21:30:34.279 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2024-03-13T21:30:34.518 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2024-03-13T21:30:34.519 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2024-03-13T21:30:34.738 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:00:51 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:30:35.207 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2024-03-13T21:30:35.255 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:15.2.17-0.el8.x86_64 1/26 2024-03-13T21:30:35.255 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:15.2.17-0.el8.x86_64 2/26 2024-03-13T21:30:35.255 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:15.2.17-0.el8.x86_64 3/26 2024-03-13T21:30:35.256 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:15.2.17-0.el8.x86_64 4/26 2024-03-13T21:30:35.256 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:15.2.17-0.el8.x86_64 5/26 2024-03-13T21:30:35.256 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:15.2.17-0.el8.x86_64 6/26 2024-03-13T21:30:35.256 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:15.2.17-0.el8.x86_64 7/26 2024-03-13T21:30:35.256 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:15.2.17-0.el8.x86_64 8/26 2024-03-13T21:30:35.257 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:15.2.17-0.el8.x86_64 9/26 2024-03-13T21:30:35.257 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:15.2.17-0.el8.x86_64 10/26 2024-03-13T21:30:35.257 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:15.2.17-0.el8.x86_64 11/26 2024-03-13T21:30:35.257 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:15.2.17-0.el8.x86_64 12/26 2024-03-13T21:30:35.257 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:15.2.17-0.el8.x86_64 13/26 2024-03-13T21:30:35.257 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/26 2024-03-13T21:30:35.258 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/26 2024-03-13T21:30:35.258 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/26 2024-03-13T21:30:35.258 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/26 2024-03-13T21:30:35.258 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 18/26 2024-03-13T21:30:35.258 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 19/26 2024-03-13T21:30:35.258 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : leveldb-1.22-1.el8.x86_64 20/26 2024-03-13T21:30:35.258 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 21/26 2024-03-13T21:30:35.258 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 22/26 2024-03-13T21:30:35.258 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:15.2.17-0.el8.x86_64 23/26 2024-03-13T21:30:35.258 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2024-03-13T21:30:35.259 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:15.2.17-0.el8.x86_64 25/26 2024-03-13T21:30:35.264 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:30:35.482 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-03-13T21:30:35.721 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-03-13T21:30:35.960 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-03-13T21:30:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2024-03-13T21:30:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:36.111 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-03-13T21:30:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:15.2.17-0.el8.x86_64 librbd1-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.111 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:30:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.111 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: fmt-6.2.1-1.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: leveldb-1.22-1.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.112 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-13T21:30:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:36.113 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:36.113 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:30:36.209 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-03-13T21:30:36.415 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-test 2024-03-13T21:30:36.431 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-03-13T21:30:36.450 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:30:36.454 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:30:36.454 INFO:teuthology.orchestra.run.smithi023.stdout: Package Arch Version Repository Size 2024-03-13T21:30:36.454 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:30:36.454 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:30:36.454 INFO:teuthology.orchestra.run.smithi023.stdout: ceph x86_64 2:15.2.17-0.el8 ceph 6.3 k 2024-03-13T21:30:36.454 INFO:teuthology.orchestra.run.smithi023.stdout:Installing dependencies: 2024-03-13T21:30:36.454 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mds x86_64 2:15.2.17-0.el8 ceph 2.0 M 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr x86_64 2:15.2.17-0.el8 ceph 1.4 M 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-modules-core noarch 2:15.2.17-0.el8 ceph-noarch 193 k 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mon x86_64 2:15.2.17-0.el8 ceph 4.3 M 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-osd x86_64 2:15.2.17-0.el8 ceph 17 M 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-13T21:30:36.455 INFO:teuthology.orchestra.run.smithi023.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-13T21:30:36.456 INFO:teuthology.orchestra.run.smithi023.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-13T21:30:36.457 INFO:teuthology.orchestra.run.smithi023.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-03-13T21:30:36.457 INFO:teuthology.orchestra.run.smithi023.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-13T21:30:36.457 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:36.457 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:30:36.457 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:30:36.457 INFO:teuthology.orchestra.run.smithi023.stdout:Install 32 Packages 2024-03-13T21:30:36.457 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:36.458 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 28 M 2024-03-13T21:30:36.458 INFO:teuthology.orchestra.run.smithi023.stdout:Installed size: 91 M 2024-03-13T21:30:36.458 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:30:36.595 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-03-13T21:30:36.797 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-03-13T21:30:36.991 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-03-13T21:30:37.190 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-03-13T21:30:37.198 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:41 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:30:37.256 INFO:teuthology.orchestra.run.smithi023.stdout:(1/32): ceph-15.2.17-0.el8.x86_64.rpm 116 kB/s | 6.3 kB 00:00 2024-03-13T21:30:37.315 INFO:teuthology.orchestra.run.smithi023.stdout:(2/32): ceph-mgr-15.2.17-0.el8.x86_64.rpm 13 MB/s | 1.4 MB 00:00 2024-03-13T21:30:37.357 INFO:teuthology.orchestra.run.smithi023.stdout:(3/32): ceph-mds-15.2.17-0.el8.x86_64.rpm 13 MB/s | 2.0 MB 00:00 2024-03-13T21:30:37.382 INFO:teuthology.orchestra.run.smithi023.stdout:(4/32): ceph-mgr-modules-core-15.2.17-0.el8.noa 7.6 MB/s | 193 kB 00:00 2024-03-13T21:30:37.437 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-03-13T21:30:37.457 INFO:teuthology.orchestra.run.smithi023.stdout:(5/32): ceph-mon-15.2.17-0.el8.x86_64.rpm 22 MB/s | 4.3 MB 00:00 2024-03-13T21:30:37.541 INFO:teuthology.orchestra.run.smithi023.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 466 kB/s | 39 kB 00:00 2024-03-13T21:30:37.566 INFO:teuthology.orchestra.run.smithi023.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 854 kB/s | 157 kB 00:00 2024-03-13T21:30:37.693 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-03-13T21:30:37.758 INFO:teuthology.orchestra.run.smithi023.stdout:(8/32): ceph-osd-15.2.17-0.el8.x86_64.rpm 38 MB/s | 17 MB 00:00 2024-03-13T21:30:37.783 INFO:teuthology.orchestra.run.smithi023.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 427 kB/s | 103 kB 00:00 2024-03-13T21:30:37.809 INFO:teuthology.orchestra.run.smithi023.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 235 kB/s | 57 kB 00:00 2024-03-13T21:30:37.934 INFO:teuthology.orchestra.run.smithi023.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 2.5 MB/s | 457 kB 00:00 2024-03-13T21:30:37.941 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-03-13T21:30:38.035 INFO:teuthology.orchestra.run.smithi023.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.1 MB/s | 249 kB 00:00 2024-03-13T21:30:38.110 INFO:teuthology.orchestra.run.smithi023.stdout:(13/32): python3-cryptography-3.2.1-7.el8.x86_6 2.1 MB/s | 641 kB 00:00 2024-03-13T21:30:38.143 INFO:teuthology.orchestra.run.smithi023.stdout:(14/32): python3-pycparser-2.14-14.el8.noarch.r 522 kB/s | 109 kB 00:00 2024-03-13T21:30:38.318 INFO:teuthology.orchestra.run.smithi023.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 910 kB/s | 185 kB 00:00 2024-03-13T21:30:38.336 INFO:teuthology.orchestra.run.smithi023.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 926 kB/s | 173 kB 00:00 2024-03-13T21:30:38.344 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-03-13T21:30:38.378 INFO:teuthology.orchestra.run.smithi023.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 267 kB/s | 11 kB 00:00 2024-03-13T21:30:38.419 INFO:teuthology.orchestra.run.smithi023.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 424 kB/s | 18 kB 00:00 2024-03-13T21:30:38.491 INFO:teuthology.orchestra.run.smithi023.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 684 kB/s | 49 kB 00:00 2024-03-13T21:30:38.561 INFO:teuthology.orchestra.run.smithi023.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.5 MB/s | 384 kB 00:00 2024-03-13T21:30:38.587 INFO:teuthology.orchestra.run.smithi023.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 618 kB/s | 59 kB 00:00 2024-03-13T21:30:38.588 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-03-13T21:30:38.612 INFO:teuthology.orchestra.run.smithi023.stdout:(22/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 75 kB/s | 44 kB 00:00 2024-03-13T21:30:38.638 INFO:teuthology.orchestra.run.smithi023.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2024-03-13T21:30:38.679 INFO:teuthology.orchestra.run.smithi023.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 281 kB/s | 19 kB 00:00 2024-03-13T21:30:38.721 INFO:teuthology.orchestra.run.smithi023.stdout:(25/32): python3-tempora-1.14.1-5.el8.noarch.rp 703 kB/s | 29 kB 00:00 2024-03-13T21:30:38.746 INFO:teuthology.orchestra.run.smithi023.stdout:(26/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.5 MB/s | 283 kB 00:00 2024-03-13T21:30:38.772 INFO:teuthology.orchestra.run.smithi023.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 549 kB/s | 27 kB 00:00 2024-03-13T21:30:38.776 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-03-13T21:30:38.814 INFO:teuthology.orchestra.run.smithi023.stdout:(28/32): python3-singledispatch-3.4.0.3-18.el8. 136 kB/s | 24 kB 00:00 2024-03-13T21:30:38.900 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:30:38.901 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:30:38.901 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-03-13T21:30:38.901 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:30:38.901 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:30:38.901 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:15.2.17-0.el8 ceph 42 M 2024-03-13T21:30:38.901 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-03-13T21:30:38.901 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-03-13T21:30:38.901 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-03-13T21:30:38.902 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-03-13T21:30:38.902 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-03-13T21:30:38.902 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:38.902 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:30:38.902 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:30:38.902 INFO:teuthology.orchestra.run.smithi064.stdout:Install 5 Packages 2024-03-13T21:30:38.902 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:38.903 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 43 M 2024-03-13T21:30:38.903 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 155 M 2024-03-13T21:30:38.903 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:30:38.906 INFO:teuthology.orchestra.run.smithi023.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.6 MB/s | 255 kB 00:00 2024-03-13T21:30:38.972 INFO:teuthology.orchestra.run.smithi023.stdout:(30/32): python3-zc-lockfile-2.0-2.el8.noarch.r 345 kB/s | 23 kB 00:00 2024-03-13T21:30:39.008 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-03-13T21:30:39.014 INFO:teuthology.orchestra.run.smithi023.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.0 MB/s | 251 kB 00:00 2024-03-13T21:30:39.102 INFO:teuthology.orchestra.run.smithi064.stdout:(1/5): jq-1.6-8.el8.x86_64.rpm 1.3 MB/s | 203 kB 00:00 2024-03-13T21:30:39.127 INFO:teuthology.orchestra.run.smithi064.stdout:(2/5): oniguruma-6.8.2-3.el8.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-03-13T21:30:39.139 INFO:teuthology.orchestra.run.smithi023.stdout:(32/32): python3-webtest-2.0.33-1.el8.noarch.rp 262 kB/s | 85 kB 00:00 2024-03-13T21:30:39.144 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:30:39.144 INFO:teuthology.orchestra.run.smithi023.stdout:Total 11 MB/s | 28 MB 00:02 2024-03-13T21:30:39.169 INFO:teuthology.orchestra.run.smithi064.stdout:(3/5): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-13T21:30:39.203 INFO:teuthology.orchestra.run.smithi064.stdout:(4/5): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2024-03-13T21:30:39.230 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:30:39.236 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-13T21:30:39.272 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:30:39.272 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:30:39.560 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-03-13T21:30:39.586 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:30:39.590 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:30:39.667 INFO:teuthology.orchestra.run.smithi064.stdout:(5/5): ceph-test-15.2.17-0.el8.x86_64.rpm 58 MB/s | 42 MB 00:00 2024-03-13T21:30:39.668 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:30:39.668 INFO:teuthology.orchestra.run.smithi064.stdout:Total 56 MB/s | 43 MB 00:00 2024-03-13T21:30:39.710 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:30:39.731 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:30:39.732 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:30:39.791 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-03-13T21:30:40.029 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-03-13T21:30:40.062 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:30:40.066 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:30:40.198 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-03-13T21:30:40.486 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:30:40.517 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-03-13T21:30:40.728 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/5 2024-03-13T21:30:40.739 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-03-13T21:30:40.746 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:30:40.945 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/5 2024-03-13T21:30:40.964 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-03-13T21:30:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-03-13T21:30:41.058 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/5 2024-03-13T21:30:41.135 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/5 2024-03-13T21:30:41.137 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-03-13T21:30:41.335 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-03-13T21:30:41.359 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-03-13T21:30:41.578 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-03-13T21:30:41.653 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-03-13T21:30:41.783 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-03-13T21:30:41.965 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-03-13T21:30:42.007 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 27/32 2024-03-13T21:30:42.174 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-03-13T21:30:42.196 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2024-03-13T21:30:42.377 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-03-13T21:30:42.606 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-03-13T21:30:42.880 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-03-13T21:30:43.122 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-03-13T21:30:43.371 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-03-13T21:30:43.555 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2024-03-13T21:30:43.758 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-03-13T21:30:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2024-03-13T21:30:43.991 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-03-13T21:30:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2024-03-13T21:30:44.221 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-03-13T21:30:44.344 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2024-03-13T21:30:44.441 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-03-13T21:30:44.525 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2024-03-13T21:30:44.630 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-13T21:30:44.702 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2024-03-13T21:30:44.719 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2024-03-13T21:30:44.854 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:15.2.17-0.el8.x86_64 32/32 2024-03-13T21:30:44.961 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-03-13T21:30:45.152 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-03-13T21:30:45.229 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-8.el8.x86_64 4/5 2024-03-13T21:30:45.259 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-2:15.2.17-0.el8.x86_64 32/32 2024-03-13T21:30:45.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:15.2.17-0.el8.x86_64 1/32 2024-03-13T21:30:45.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:15.2.17-0.el8.x86_64 2/32 2024-03-13T21:30:45.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:15.2.17-0.el8.x86_64 3/32 2024-03-13T21:30:45.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:15.2.17-0.el8.x86_64 4/32 2024-03-13T21:30:45.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:15.2.17-0.el8.x86_64 5/32 2024-03-13T21:30:45.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 6/32 2024-03-13T21:30:45.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-03-13T21:30:45.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-03-13T21:30:45.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-03-13T21:30:45.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-03-13T21:30:45.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-03-13T21:30:45.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-03-13T21:30:45.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-03-13T21:30:45.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-03-13T21:30:45.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-03-13T21:30:45.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-03-13T21:30:45.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-03-13T21:30:45.365 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-03-13T21:30:45.522 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:15.2.17-0.el8.x86_64 5/5 2024-03-13T21:30:45.525 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-03-13T21:30:45.812 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-03-13T21:30:45.819 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-test-2:15.2.17-0.el8.x86_64 5/5 2024-03-13T21:30:45.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:15.2.17-0.el8.x86_64 1/5 2024-03-13T21:30:45.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-8.el8.x86_64 2/5 2024-03-13T21:30:45.819 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 3/5 2024-03-13T21:30:45.820 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/5 2024-03-13T21:30:46.017 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-03-13T21:30:46.219 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 5/5 2024-03-13T21:30:46.219 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:46.219 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:30:46.219 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:15.2.17-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-03-13T21:30:46.219 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.8.2-3.el8.x86_64 socat-1.7.4.1-1.el8.x86_64 2024-03-13T21:30:46.219 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-13T21:30:46.219 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:46.219 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:30:46.234 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-03-13T21:30:46.234 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:46.234 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:30:46.234 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:46.234 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:46.234 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-13T21:30:46.235 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-2.6-1.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-13T21:30:46.236 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-13T21:30:46.237 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-13T21:30:46.237 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:46.237 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:30:46.269 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-03-13T21:30:46.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph 2024-03-13T21:30:46.515 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-base 2024-03-13T21:30:46.613 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-03-13T21:30:46.886 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-03-13T21:30:47.205 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 27/32 2024-03-13T21:30:47.231 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:51 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:30:47.272 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:08 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:30:47.420 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2024-03-13T21:30:48.810 INFO:teuthology.orchestra.run.smithi077.stdout:Package ceph-base-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:30:48.856 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:30:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-03-13T21:30:48.857 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:30:48.866 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2024-03-13T21:30:48.945 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:30:48.949 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:30:48.949 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-03-13T21:30:48.949 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:30:48.949 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:30:48.949 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:15.2.17-0.el8 ceph 6.3 k 2024-03-13T21:30:48.949 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-03-13T21:30:48.949 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:15.2.17-0.el8 ceph 2.0 M 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:15.2.17-0.el8 ceph 1.4 M 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:15.2.17-0.el8 ceph-noarch 193 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:15.2.17-0.el8 ceph 4.3 M 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:15.2.17-0.el8 ceph 17 M 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-03-13T21:30:48.950 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-13T21:30:48.951 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-03-13T21:30:48.952 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-13T21:30:48.952 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:48.952 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:30:48.952 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:30:48.952 INFO:teuthology.orchestra.run.smithi064.stdout:Install 32 Packages 2024-03-13T21:30:48.952 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:30:48.952 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 28 M 2024-03-13T21:30:48.952 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 91 M 2024-03-13T21:30:48.953 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:30:48.964 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-immutable-object-cache 2024-03-13T21:30:49.045 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2024-03-13T21:30:49.403 INFO:teuthology.orchestra.run.smithi064.stdout:(1/32): ceph-15.2.17-0.el8.x86_64.rpm 132 kB/s | 6.3 kB 00:00 2024-03-13T21:30:49.453 INFO:teuthology.orchestra.run.smithi064.stdout:(2/32): ceph-mgr-15.2.17-0.el8.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-03-13T21:30:49.455 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2024-03-13T21:30:49.503 INFO:teuthology.orchestra.run.smithi064.stdout:(3/32): ceph-mds-15.2.17-0.el8.x86_64.rpm 14 MB/s | 2.0 MB 00:00 2024-03-13T21:30:49.562 INFO:teuthology.orchestra.run.smithi064.stdout:(4/32): ceph-mgr-modules-core-15.2.17-0.el8.noa 3.2 MB/s | 193 kB 00:00 2024-03-13T21:30:49.637 INFO:teuthology.orchestra.run.smithi064.stdout:(5/32): ceph-mon-15.2.17-0.el8.x86_64.rpm 19 MB/s | 4.3 MB 00:00 2024-03-13T21:30:49.667 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2024-03-13T21:30:49.716 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:10 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:30:49.860 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2024-03-13T21:30:49.879 INFO:teuthology.orchestra.run.smithi064.stdout:(6/32): ceph-osd-15.2.17-0.el8.x86_64.rpm 39 MB/s | 17 MB 00:00 2024-03-13T21:30:49.905 INFO:teuthology.orchestra.run.smithi064.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 145 kB/s | 39 kB 00:00 2024-03-13T21:30:49.930 INFO:teuthology.orchestra.run.smithi064.stdout:(8/32): python3-mako-1.0.6-14.el8.noarch.rpm 427 kB/s | 157 kB 00:00 2024-03-13T21:30:49.955 INFO:teuthology.orchestra.run.smithi064.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 1.3 MB/s | 103 kB 00:00 2024-03-13T21:30:50.006 INFO:teuthology.orchestra.run.smithi064.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 567 kB/s | 57 kB 00:00 2024-03-13T21:30:50.020 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2024-03-13T21:30:50.031 INFO:teuthology.orchestra.run.smithi064.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 4.4 MB/s | 457 kB 00:00 2024-03-13T21:30:50.038 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2024-03-13T21:30:50.195 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-2:15.2.17-0.el8.x86_64 32/32 2024-03-13T21:30:50.198 INFO:teuthology.orchestra.run.smithi064.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.0 MB/s | 249 kB 00:00 2024-03-13T21:30:50.223 INFO:teuthology.orchestra.run.smithi064.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 568 kB/s | 109 kB 00:00 2024-03-13T21:30:50.307 INFO:teuthology.orchestra.run.smithi064.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 2.1 MB/s | 641 kB 00:00 2024-03-13T21:30:50.374 INFO:teuthology.orchestra.run.smithi064.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 248 kB/s | 44 kB 00:00 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-2:15.2.17-0.el8.x86_64 32/32 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-2:15.2.17-0.el8.x86_64 1/32 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mds-2:15.2.17-0.el8.x86_64 2/32 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mgr-2:15.2.17-0.el8.x86_64 3/32 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mon-2:15.2.17-0.el8.x86_64 4/32 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-osd-2:15.2.17-0.el8.x86_64 5/32 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 6/32 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-03-13T21:30:50.615 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-03-13T21:30:50.616 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-03-13T21:30:50.617 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-03-13T21:30:50.617 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-03-13T21:30:50.617 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-03-13T21:30:50.617 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-03-13T21:30:50.617 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-03-13T21:30:50.617 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-03-13T21:30:51.311 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:30:51.312 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:51.312 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repo Size 2024-03-13T21:30:51.312 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:51.312 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:30:51.312 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:15.2.17-0.el8 ceph 203 k 2024-03-13T21:30:51.312 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:51.313 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:30:51.313 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:51.313 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-03-13T21:30:51.313 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:51.313 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 203 k 2024-03-13T21:30:51.313 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 599 k 2024-03-13T21:30:51.313 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:30:51.366 INFO:teuthology.orchestra.run.smithi077.stdout:ceph-immutable-object-cache-15.2.17-0.el8.x86_6 3.8 MB/s | 203 kB 00:00 2024-03-13T21:30:51.366 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:30:51.366 INFO:teuthology.orchestra.run.smithi077.stdout:Total 3.7 MB/s | 203 kB 00:00 2024-03-13T21:30:51.367 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:30:51.377 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:30:51.377 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:30:51.382 INFO:teuthology.orchestra.run.smithi064.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 160 kB/s | 185 kB 00:01 2024-03-13T21:30:51.425 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:30:51.429 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:30:51.441 INFO:teuthology.orchestra.run.smithi064.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 360 kB/s | 384 kB 00:01 2024-03-13T21:30:51.447 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-03-13T21:30:51.447 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mds-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-mon-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-osd-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-13T21:30:51.448 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-portend-2.6-1.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-13T21:30:51.449 INFO:teuthology.orchestra.run.smithi023.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-13T21:30:51.450 INFO:teuthology.orchestra.run.smithi023.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-13T21:30:51.450 INFO:teuthology.orchestra.run.smithi023.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-13T21:30:51.450 INFO:teuthology.orchestra.run.smithi023.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-13T21:30:51.450 INFO:teuthology.orchestra.run.smithi023.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-13T21:30:51.450 INFO:teuthology.orchestra.run.smithi023.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-13T21:30:51.450 INFO:teuthology.orchestra.run.smithi023.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-13T21:30:51.450 INFO:teuthology.orchestra.run.smithi023.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-13T21:30:51.450 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:51.450 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:30:51.491 INFO:teuthology.orchestra.run.smithi064.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2024-03-13T21:30:51.539 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:30:51.625 INFO:teuthology.orchestra.run.smithi064.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 96 kB/s | 18 kB 00:00 2024-03-13T21:30:51.692 INFO:teuthology.orchestra.run.smithi064.stdout:(20/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 125 kB/s | 173 kB 00:01 2024-03-13T21:30:51.709 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install ceph-base 2024-03-13T21:30:51.710 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:30:51.833 INFO:teuthology.orchestra.run.smithi064.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 281 kB/s | 59 kB 00:00 2024-03-13T21:30:51.859 INFO:teuthology.orchestra.run.smithi064.stdout:(22/32): python3-logutils-0.3.5-11.el8.noarch.r 133 kB/s | 49 kB 00:00 2024-03-13T21:30:52.142 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:30:52.506 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:09 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:30:52.528 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:30:52.528 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:52.528 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:30:52.528 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:52.528 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:52.529 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:30:52.551 INFO:teuthology.orchestra.run.smithi064.stdout:(23/32): python3-simplegeneric-0.8.1-17.el8.noa 27 kB/s | 19 kB 00:00 2024-03-13T21:30:52.753 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-mgr 2024-03-13T21:30:52.934 INFO:teuthology.orchestra.run.smithi064.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 228 kB/s | 283 kB 00:01 2024-03-13T21:30:53.109 INFO:teuthology.orchestra.run.smithi064.stdout:(25/32): python3-portend-2.6-1.el8.noarch.rpm 12 kB/s | 16 kB 00:01 2024-03-13T21:30:53.143 INFO:teuthology.orchestra.run.smithi064.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 140 kB/s | 29 kB 00:00 2024-03-13T21:30:53.177 INFO:teuthology.orchestra.run.smithi064.stdout:(27/32): python3-singledispatch-3.4.0.3-18.el8. 38 kB/s | 24 kB 00:00 2024-03-13T21:30:53.485 INFO:teuthology.orchestra.run.smithi064.stdout:(28/32): python3-waitress-1.4.3-1.el8.noarch.rp 747 kB/s | 255 kB 00:00 2024-03-13T21:30:53.516 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:14 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:30:53.652 INFO:teuthology.orchestra.run.smithi064.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 512 kB/s | 85 kB 00:00 2024-03-13T21:30:53.753 INFO:teuthology.orchestra.run.smithi064.stdout:(30/32): python3-zc-lockfile-2.0-2.el8.noarch.r 229 kB/s | 23 kB 00:00 2024-03-13T21:30:54.136 INFO:teuthology.orchestra.run.smithi064.stdout:(31/32): python3-trustme-0.6.0-4.el8.noarch.rpm 27 kB/s | 27 kB 00:01 2024-03-13T21:30:54.167 INFO:teuthology.orchestra.run.smithi023.stdout:Package ceph-base-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:30:54.213 INFO:teuthology.orchestra.run.smithi064.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 242 kB/s | 251 kB 00:01 2024-03-13T21:30:54.214 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:30:54.214 INFO:teuthology.orchestra.run.smithi064.stdout:Total 5.4 MB/s | 28 MB 00:05 2024-03-13T21:30:54.217 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:30:54.218 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2024-03-13T21:30:54.218 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:30:54.298 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:30:54.325 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install ceph-immutable-object-cache 2024-03-13T21:30:54.339 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:30:54.339 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:30:54.663 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:30:54.666 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:30:55.078 INFO:teuthology.orchestra.run.smithi077.stdout:Package ceph-mgr-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:30:55.115 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:12 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:30:55.124 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:30:55.125 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-03-13T21:30:55.126 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:30:55.680 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-fuse 2024-03-13T21:30:55.823 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:30:56.066 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-03-13T21:30:56.297 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-03-13T21:30:56.431 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:17 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:30:56.502 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-03-13T21:30:56.762 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-03-13T21:30:56.831 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:30:56.831 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:30:56.831 INFO:teuthology.orchestra.run.smithi023.stdout: Package Arch Version Repo Size 2024-03-13T21:30:56.832 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:30:56.832 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:30:56.832 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-immutable-object-cache x86_64 2:15.2.17-0.el8 ceph 203 k 2024-03-13T21:30:56.832 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:56.832 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:30:56.832 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:30:56.832 INFO:teuthology.orchestra.run.smithi023.stdout:Install 1 Package 2024-03-13T21:30:56.832 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:56.833 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 203 k 2024-03-13T21:30:56.833 INFO:teuthology.orchestra.run.smithi023.stdout:Installed size: 599 k 2024-03-13T21:30:56.833 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:30:56.881 INFO:teuthology.orchestra.run.smithi023.stdout:ceph-immutable-object-cache-15.2.17-0.el8.x86_6 4.1 MB/s | 203 kB 00:00 2024-03-13T21:30:56.881 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:30:56.882 INFO:teuthology.orchestra.run.smithi023.stdout:Total 4.0 MB/s | 203 kB 00:00 2024-03-13T21:30:56.882 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:30:56.894 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:30:56.894 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:30:56.945 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:30:56.948 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:30:56.959 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-03-13T21:30:57.064 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:30:57.175 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-03-13T21:30:57.252 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:30:57.341 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-03-13T21:30:57.561 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-03-13T21:30:57.676 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:30:57.798 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-03-13T21:30:58.009 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:15.2.17-0.el8 ceph 572 k 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout:Install 2 Packages 2024-03-13T21:30:58.010 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:30:58.011 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 656 k 2024-03-13T21:30:58.011 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 1.8 M 2024-03-13T21:30:58.011 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:30:58.032 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:30:58.032 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:58.032 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:30:58.032 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 2024-03-13T21:30:58.032 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:30:58.032 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:30:58.054 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-03-13T21:30:58.236 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install ceph-mgr 2024-03-13T21:30:58.256 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-03-13T21:30:58.497 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-03-13T21:30:58.738 INFO:teuthology.orchestra.run.smithi077.stdout:(1/2): ceph-fuse-15.2.17-0.el8.x86_64.rpm 12 MB/s | 572 kB 00:00 2024-03-13T21:30:58.890 INFO:teuthology.orchestra.run.smithi077.stdout:(2/2): fuse-2.9.7-19.el8.x86_64.rpm 422 kB/s | 83 kB 00:00 2024-03-13T21:30:58.890 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:30:58.890 INFO:teuthology.orchestra.run.smithi077.stdout:Total 747 kB/s | 656 kB 00:00 2024-03-13T21:30:58.900 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:30:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-03-13T21:30:58.922 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:30:58.923 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:30:59.012 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:30:59.015 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:30:59.024 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:16 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:30:59.175 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-03-13T21:30:59.184 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:30:59.363 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-03-13T21:30:59.429 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-03-13T21:30:59.599 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2024-03-13T21:30:59.615 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-03-13T21:30:59.839 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-13T21:30:59.996 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2024-03-13T21:30:59.996 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:15.2.17-0.el8.x86_64 1/2 2024-03-13T21:31:00.161 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-03-13T21:31:00.377 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-03-13T21:31:00.386 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-03-13T21:31:00.386 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:00.386 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:31:00.386 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:15.2.17-0.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-03-13T21:31:00.386 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:00.386 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:00.574 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-03-13T21:31:00.620 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install librados-devel 2024-03-13T21:31:00.679 INFO:teuthology.orchestra.run.smithi023.stdout:Package ceph-mgr-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:00.728 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:00.729 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2024-03-13T21:31:00.729 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:00.785 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-03-13T21:31:00.920 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install ceph-fuse 2024-03-13T21:31:01.114 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-03-13T21:31:01.334 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-03-13T21:31:01.382 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:22 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:01.579 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-03-13T21:31:01.708 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:18 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:01.938 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-03-13T21:31:02.244 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-03-13T21:31:02.611 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 27/32 2024-03-13T21:31:02.791 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2024-03-13T21:31:02.977 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:02.978 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:02.978 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:02.978 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:02.978 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:31:02.979 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:15.2.17-0.el8 ceph 140 k 2024-03-13T21:31:02.979 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:02.979 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:31:02.979 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:02.979 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-03-13T21:31:02.979 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:02.981 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 140 k 2024-03-13T21:31:02.981 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 478 k 2024-03-13T21:31:02.981 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:31:03.041 INFO:teuthology.orchestra.run.smithi077.stdout:librados-devel-15.2.17-0.el8.x86_64.rpm 2.2 MB/s | 140 kB 00:00 2024-03-13T21:31:03.042 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:03.042 INFO:teuthology.orchestra.run.smithi077.stdout:Total 2.2 MB/s | 140 kB 00:00 2024-03-13T21:31:03.042 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:31:03.047 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:31:03.047 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:31:03.093 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:31:03.096 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:31:03.199 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:31:03.415 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:03.416 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:03.416 INFO:teuthology.orchestra.run.smithi023.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:03.416 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:03.416 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:31:03.416 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-fuse x86_64 2:15.2.17-0.el8 ceph 572 k 2024-03-13T21:31:03.417 INFO:teuthology.orchestra.run.smithi023.stdout:Installing dependencies: 2024-03-13T21:31:03.417 INFO:teuthology.orchestra.run.smithi023.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-03-13T21:31:03.417 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:03.417 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:31:03.417 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:03.417 INFO:teuthology.orchestra.run.smithi023.stdout:Install 2 Packages 2024-03-13T21:31:03.417 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:03.417 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 656 k 2024-03-13T21:31:03.418 INFO:teuthology.orchestra.run.smithi023.stdout:Installed size: 1.8 M 2024-03-13T21:31:03.418 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:31:03.431 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:03.720 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:03.809 INFO:teuthology.orchestra.run.smithi023.stdout:(1/2): ceph-fuse-15.2.17-0.el8.x86_64.rpm 4.3 MB/s | 572 kB 00:00 2024-03-13T21:31:04.064 INFO:teuthology.orchestra.run.smithi023.stdout:(2/2): fuse-2.9.7-19.el8.x86_64.rpm 217 kB/s | 83 kB 00:00 2024-03-13T21:31:04.065 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:04.065 INFO:teuthology.orchestra.run.smithi023.stdout:Total 1.0 MB/s | 656 kB 00:00 2024-03-13T21:31:04.076 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:31:04.098 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:31:04.098 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:31:04.108 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:04.108 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:04.108 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:31:04.108 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:04.108 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:04.108 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:04.189 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:31:04.192 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:31:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:15.2.17-0.el8.x86_64 28/32 2024-03-13T21:31:04.347 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install libcephfs2 2024-03-13T21:31:04.362 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:31:04.437 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2024-03-13T21:31:04.600 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-03-13T21:31:04.751 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2024-03-13T21:31:04.846 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:15.2.17-0.el8.x86_64 29/32 2024-03-13T21:31:05.070 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2024-03-13T21:31:05.109 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:26 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:05.150 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2024-03-13T21:31:05.150 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : ceph-fuse-2:15.2.17-0.el8.x86_64 1/2 2024-03-13T21:31:05.263 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:15.2.17-0.el8.x86_64 30/32 2024-03-13T21:31:05.404 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2024-03-13T21:31:05.422 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:15.2.17-0.el8.x86_64 31/32 2024-03-13T21:31:05.543 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:15.2.17-0.el8.x86_64 32/32 2024-03-13T21:31:05.554 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-03-13T21:31:05.554 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:05.554 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:31:05.554 INFO:teuthology.orchestra.run.smithi023.stdout: ceph-fuse-2:15.2.17-0.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-03-13T21:31:05.554 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:05.554 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:05.804 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install librados-devel 2024-03-13T21:31:06.001 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-2:15.2.17-0.el8.x86_64 32/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:15.2.17-0.el8.x86_64 1/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:15.2.17-0.el8.x86_64 2/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:15.2.17-0.el8.x86_64 3/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:15.2.17-0.el8.x86_64 4/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:15.2.17-0.el8.x86_64 5/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 6/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-03-13T21:31:06.002 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-03-13T21:31:06.003 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-03-13T21:31:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-03-13T21:31:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-03-13T21:31:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-03-13T21:31:06.004 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-03-13T21:31:06.599 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:23 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:06.627 INFO:teuthology.orchestra.run.smithi077.stdout:Package libcephfs2-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:06.673 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:06.674 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-03-13T21:31:06.674 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:06.786 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install libcephfs-devel 2024-03-13T21:31:06.926 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-03-13T21:31:06.926 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:06.926 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:31:06.926 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:06.926 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:06.926 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:06.926 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:15.2.17-0.el8.noarch 2024-03-13T21:31:06.926 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-13T21:31:06.927 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-2.6-1.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:06.928 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:07.244 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-base 2024-03-13T21:31:07.541 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:28 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:08.036 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:12 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:08.323 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:08.323 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:08.323 INFO:teuthology.orchestra.run.smithi023.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:08.323 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:08.323 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:31:08.324 INFO:teuthology.orchestra.run.smithi023.stdout: librados-devel x86_64 2:15.2.17-0.el8 ceph 140 k 2024-03-13T21:31:08.324 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:08.324 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:31:08.324 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:08.324 INFO:teuthology.orchestra.run.smithi023.stdout:Install 1 Package 2024-03-13T21:31:08.324 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:08.324 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 140 k 2024-03-13T21:31:08.324 INFO:teuthology.orchestra.run.smithi023.stdout:Installed size: 478 k 2024-03-13T21:31:08.325 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:31:08.371 INFO:teuthology.orchestra.run.smithi023.stdout:librados-devel-15.2.17-0.el8.x86_64.rpm 3.0 MB/s | 140 kB 00:00 2024-03-13T21:31:08.371 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:08.372 INFO:teuthology.orchestra.run.smithi023.stdout:Total 2.9 MB/s | 140 kB 00:00 2024-03-13T21:31:08.372 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:31:08.378 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:31:08.378 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:31:08.430 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:31:08.433 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:31:08.549 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:31:08.724 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:09.035 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: librados-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:09.130 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:09.131 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:09.131 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:09.131 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:09.131 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:31:09.131 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:15.2.17-0.el8 ceph 22 k 2024-03-13T21:31:09.131 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:09.131 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:31:09.131 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:09.131 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-03-13T21:31:09.131 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:09.132 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 22 k 2024-03-13T21:31:09.132 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 78 k 2024-03-13T21:31:09.132 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:31:09.186 INFO:teuthology.orchestra.run.smithi077.stdout:libcephfs-devel-15.2.17-0.el8.x86_64.rpm 402 kB/s | 22 kB 00:00 2024-03-13T21:31:09.187 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:09.187 INFO:teuthology.orchestra.run.smithi077.stdout:Total 393 kB/s | 22 kB 00:00 2024-03-13T21:31:09.187 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:31:09.191 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:31:09.191 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:31:09.192 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:31:09.195 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:31:09.243 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:31:09.408 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:09.408 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:09.409 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:31:09.409 INFO:teuthology.orchestra.run.smithi023.stdout: librados-devel-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:09.409 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:09.409 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:09.439 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:09.615 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install libcephfs2 2024-03-13T21:31:09.688 INFO:teuthology.orchestra.run.smithi064.stdout:Package ceph-base-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:09.693 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:09.739 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:09.740 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-03-13T21:31:09.740 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:09.846 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-immutable-object-cache 2024-03-13T21:31:10.039 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:10.040 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:10.040 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:31:10.040 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:10.040 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:10.040 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:10.156 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install librados2 2024-03-13T21:31:10.403 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:27 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:10.633 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:14 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:10.905 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:31 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:12.058 INFO:teuthology.orchestra.run.smithi023.stdout:Package libcephfs2-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:12.108 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:12.109 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2024-03-13T21:31:12.110 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:12.219 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install libcephfs-devel 2024-03-13T21:31:12.341 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:12.341 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:12.342 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repo Size 2024-03-13T21:31:12.342 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:12.342 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:31:12.342 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:15.2.17-0.el8 ceph 203 k 2024-03-13T21:31:12.342 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:12.342 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:31:12.342 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:12.342 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-03-13T21:31:12.342 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:12.343 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 203 k 2024-03-13T21:31:12.343 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 599 k 2024-03-13T21:31:12.343 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:31:12.398 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-immutable-object-cache-15.2.17-0.el8.x86_6 3.8 MB/s | 203 kB 00:00 2024-03-13T21:31:12.398 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:12.398 INFO:teuthology.orchestra.run.smithi064.stdout:Total 3.7 MB/s | 203 kB 00:00 2024-03-13T21:31:12.398 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:31:12.408 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:31:12.408 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:31:12.457 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:12.459 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:31:12.462 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:31:12.505 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:12.506 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-03-13T21:31:12.507 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:12.583 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:31:12.617 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install librbd1 2024-03-13T21:31:12.761 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:13.004 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:30 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:13.366 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:34 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:14.119 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:31:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:14.119 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:14.119 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:14.707 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:14.707 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:14.707 INFO:teuthology.orchestra.run.smithi023.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:14.707 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:14.707 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:31:14.708 INFO:teuthology.orchestra.run.smithi023.stdout: libcephfs-devel x86_64 2:15.2.17-0.el8 ceph 22 k 2024-03-13T21:31:14.708 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:14.708 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:31:14.708 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:14.708 INFO:teuthology.orchestra.run.smithi023.stdout:Install 1 Package 2024-03-13T21:31:14.708 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:14.708 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 22 k 2024-03-13T21:31:14.708 INFO:teuthology.orchestra.run.smithi023.stdout:Installed size: 78 k 2024-03-13T21:31:14.709 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:31:14.779 INFO:teuthology.orchestra.run.smithi023.stdout:libcephfs-devel-15.2.17-0.el8.x86_64.rpm 311 kB/s | 22 kB 00:00 2024-03-13T21:31:14.779 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:14.779 INFO:teuthology.orchestra.run.smithi023.stdout:Total 306 kB/s | 22 kB 00:00 2024-03-13T21:31:14.779 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:31:14.783 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:31:14.783 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:31:14.785 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:31:14.788 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:31:14.837 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:31:14.884 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-03-13T21:31:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:14.977 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-mgr 2024-03-13T21:31:14.989 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:15.037 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-rados 2024-03-13T21:31:15.247 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:15.598 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:15.598 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:15.598 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:31:15.598 INFO:teuthology.orchestra.run.smithi023.stdout: libcephfs-devel-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:15.599 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:15.599 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:15.728 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install librados2 2024-03-13T21:31:15.765 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:19 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:15.790 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:36 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:16.520 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:33 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:17.345 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-rados-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:17.391 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:17.392 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-03-13T21:31:17.392 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:17.431 INFO:teuthology.orchestra.run.smithi064.stdout:Package ceph-mgr-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:17.481 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:17.482 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-03-13T21:31:17.482 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:17.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-fuse 2024-03-13T21:31:17.923 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-rgw 2024-03-13T21:31:18.175 INFO:teuthology.orchestra.run.smithi023.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:18.225 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:18.226 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2024-03-13T21:31:18.226 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:18.335 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install librbd1 2024-03-13T21:31:18.377 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:22 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:18.674 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:39 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:19.126 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:36 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:20.104 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:20.104 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:20.104 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:20.105 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:20.105 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:31:20.105 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:15.2.17-0.el8 ceph 572 k 2024-03-13T21:31:20.105 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-03-13T21:31:20.105 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-03-13T21:31:20.105 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:20.105 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:31:20.105 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:20.105 INFO:teuthology.orchestra.run.smithi064.stdout:Install 2 Packages 2024-03-13T21:31:20.105 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:20.106 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 656 k 2024-03-13T21:31:20.106 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 1.8 M 2024-03-13T21:31:20.106 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:31:20.242 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-rgw-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:20.282 INFO:teuthology.orchestra.run.smithi064.stdout:(1/2): ceph-fuse-15.2.17-0.el8.x86_64.rpm 11 MB/s | 572 kB 00:00 2024-03-13T21:31:20.287 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:20.289 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-03-13T21:31:20.289 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:20.398 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-cephfs 2024-03-13T21:31:20.424 INFO:teuthology.orchestra.run.smithi064.stdout:(2/2): fuse-2.9.7-19.el8.x86_64.rpm 429 kB/s | 83 kB 00:00 2024-03-13T21:31:20.424 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:20.425 INFO:teuthology.orchestra.run.smithi064.stdout:Total 2.0 MB/s | 656 kB 00:00 2024-03-13T21:31:20.435 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:31:20.457 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:31:20.457 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:31:20.544 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:31:20.548 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:31:20.728 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:31:20.787 INFO:teuthology.orchestra.run.smithi023.stdout:Package librbd1-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:20.837 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:20.839 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2024-03-13T21:31:20.839 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:20.917 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-03-13T21:31:20.945 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install python3-rados 2024-03-13T21:31:21.059 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2024-03-13T21:31:21.150 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:42 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:21.462 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-fuse-2:15.2.17-0.el8.x86_64 2/2 2024-03-13T21:31:21.463 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:15.2.17-0.el8.x86_64 1/2 2024-03-13T21:31:21.745 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:38 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:21.863 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-03-13T21:31:21.863 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:21.863 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:31:21.863 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:15.2.17-0.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-03-13T21:31:21.863 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:21.863 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:22.068 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados-devel 2024-03-13T21:31:22.662 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-cephfs-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:22.708 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:22.709 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-03-13T21:31:22.709 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:22.823 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install python3-rbd 2024-03-13T21:31:22.863 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:26 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:23.398 INFO:teuthology.orchestra.run.smithi023.stdout:Package python3-rados-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:23.449 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:23.451 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2024-03-13T21:31:23.451 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:23.553 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install python3-rgw 2024-03-13T21:31:23.570 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:44 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:24.339 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:41 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:24.573 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:24.574 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:24.574 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:24.574 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:24.574 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:31:24.574 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:15.2.17-0.el8 ceph 140 k 2024-03-13T21:31:24.574 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:24.574 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:31:24.574 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:24.575 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-03-13T21:31:24.575 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:24.575 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 140 k 2024-03-13T21:31:24.575 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 478 k 2024-03-13T21:31:24.575 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:31:24.628 INFO:teuthology.orchestra.run.smithi064.stdout:librados-devel-15.2.17-0.el8.x86_64.rpm 2.6 MB/s | 140 kB 00:00 2024-03-13T21:31:24.629 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:24.629 INFO:teuthology.orchestra.run.smithi064.stdout:Total 2.5 MB/s | 140 kB 00:00 2024-03-13T21:31:24.629 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:31:24.635 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:31:24.635 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:31:24.686 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:31:24.689 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:31:24.834 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:31:25.058 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:25.112 INFO:teuthology.orchestra.run.smithi077.stdout:Package python3-rbd-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:25.157 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:25.159 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-03-13T21:31:25.159 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:25.261 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install rbd-fuse 2024-03-13T21:31:25.358 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:25.746 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:25.747 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:31:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:25.747 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:25.747 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:25.910 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install libcephfs2 2024-03-13T21:31:25.998 INFO:teuthology.orchestra.run.smithi023.stdout:Package python3-rgw-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:26.010 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:47 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:26.048 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:26.049 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2024-03-13T21:31:26.049 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:26.155 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install python3-cephfs 2024-03-13T21:31:26.703 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:30 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:26.953 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:43 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:27.609 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:27.610 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:27.610 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:27.610 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:27.610 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:31:27.610 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:15.2.17-0.el8 ceph 73 k 2024-03-13T21:31:27.611 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:27.611 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:31:27.611 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:27.611 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-03-13T21:31:27.611 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:27.611 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 73 k 2024-03-13T21:31:27.611 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 188 k 2024-03-13T21:31:27.612 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:31:27.665 INFO:teuthology.orchestra.run.smithi077.stdout:rbd-fuse-15.2.17-0.el8.x86_64.rpm 1.3 MB/s | 73 kB 00:00 2024-03-13T21:31:27.666 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:27.666 INFO:teuthology.orchestra.run.smithi077.stdout:Total 1.3 MB/s | 73 kB 00:00 2024-03-13T21:31:27.666 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:31:27.676 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:31:27.676 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:31:27.723 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:31:27.726 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:31:27.822 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:31:28.019 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:28.304 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:28.355 INFO:teuthology.orchestra.run.smithi064.stdout:Package libcephfs2-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:28.406 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:28.408 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-03-13T21:31:28.408 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:28.504 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install libcephfs-devel 2024-03-13T21:31:28.643 INFO:teuthology.orchestra.run.smithi023.stdout:Package python3-cephfs-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:28.649 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:28.649 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:28.649 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:31:28.650 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:28.650 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:28.650 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:28.696 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:28.697 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2024-03-13T21:31:28.697 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:28.800 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install python3-rbd 2024-03-13T21:31:28.871 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install rbd-mirror 2024-03-13T21:31:29.286 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:33 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:29.586 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:46 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:29.642 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:50 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:30.997 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:30.998 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:30.998 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:30.998 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:30.998 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:31:30.998 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:15.2.17-0.el8 ceph 22 k 2024-03-13T21:31:30.998 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:30.999 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:31:30.999 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:30.999 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-03-13T21:31:30.999 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:30.999 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 22 k 2024-03-13T21:31:30.999 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 78 k 2024-03-13T21:31:30.999 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:31:31.042 INFO:teuthology.orchestra.run.smithi064.stdout:libcephfs-devel-15.2.17-0.el8.x86_64.rpm 511 kB/s | 22 kB 00:00 2024-03-13T21:31:31.043 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:31.043 INFO:teuthology.orchestra.run.smithi064.stdout:Total 496 kB/s | 22 kB 00:00 2024-03-13T21:31:31.043 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:31:31.047 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:31:31.047 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:31:31.049 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:31:31.052 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:31:31.131 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:31:31.232 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:31.233 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:31.233 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:31.233 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:31.233 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:31:31.233 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:15.2.17-0.el8 ceph 2.7 M 2024-03-13T21:31:31.233 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:31.233 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:31:31.233 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:31.234 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-03-13T21:31:31.234 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:31.234 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 2.7 M 2024-03-13T21:31:31.234 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 9.8 M 2024-03-13T21:31:31.235 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:31:31.239 INFO:teuthology.orchestra.run.smithi023.stdout:Package python3-rbd-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:31.257 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:31.291 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:31.292 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2024-03-13T21:31:31.292 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:31.318 INFO:teuthology.orchestra.run.smithi077.stdout:rbd-mirror-15.2.17-0.el8.x86_64.rpm 32 MB/s | 2.7 MB 00:00 2024-03-13T21:31:31.319 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:31.319 INFO:teuthology.orchestra.run.smithi077.stdout:Total 32 MB/s | 2.7 MB 00:00 2024-03-13T21:31:31.320 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:31:31.331 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:31:31.331 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:31:31.387 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:31:31.391 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:31:31.400 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install rbd-fuse 2024-03-13T21:31:31.530 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:31.743 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:31:31.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:31.883 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:31.883 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:31:31.883 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:31.883 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:31.883 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:31.904 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:32.000 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados2 2024-03-13T21:31:32.194 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:49 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:32.328 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:32.715 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:32.715 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:32.715 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:31:32.715 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:32.715 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:32.715 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:32.792 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:36 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:32.986 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install rbd-nbd 2024-03-13T21:31:33.748 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:54 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:33.915 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:33.915 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:33.915 INFO:teuthology.orchestra.run.smithi023.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:33.916 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:33.916 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:31:33.916 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-fuse x86_64 2:15.2.17-0.el8 ceph 73 k 2024-03-13T21:31:33.916 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:33.916 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:31:33.916 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:33.916 INFO:teuthology.orchestra.run.smithi023.stdout:Install 1 Package 2024-03-13T21:31:33.916 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:33.917 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 73 k 2024-03-13T21:31:33.917 INFO:teuthology.orchestra.run.smithi023.stdout:Installed size: 188 k 2024-03-13T21:31:33.917 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:31:33.970 INFO:teuthology.orchestra.run.smithi023.stdout:rbd-fuse-15.2.17-0.el8.x86_64.rpm 1.3 MB/s | 73 kB 00:00 2024-03-13T21:31:33.970 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:33.971 INFO:teuthology.orchestra.run.smithi023.stdout:Total 1.3 MB/s | 73 kB 00:00 2024-03-13T21:31:33.971 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:31:33.981 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:31:33.981 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:31:34.032 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:31:34.036 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:31:34.138 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:31:34.320 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:34.451 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:34.501 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:34.503 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-03-13T21:31:34.503 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:34.600 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:34.602 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librbd1 2024-03-13T21:31:34.980 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:34.980 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:34.980 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:31:34.980 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-fuse-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:34.980 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:34.980 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:35.187 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install rbd-mirror 2024-03-13T21:31:35.339 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:35.340 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:15.2.17-0.el8 ceph 140 k 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout:Install 1 Package 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 140 k 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 418 k 2024-03-13T21:31:35.341 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-03-13T21:31:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:rbd-nbd-15.2.17-0.el8.x86_64.rpm 3.6 MB/s | 140 kB 00:00 2024-03-13T21:31:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:35.381 INFO:teuthology.orchestra.run.smithi077.stdout:Total 3.4 MB/s | 140 kB 00:00 2024-03-13T21:31:35.381 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-03-13T21:31:35.390 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-03-13T21:31:35.390 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-03-13T21:31:35.391 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:39 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:35.438 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-03-13T21:31:35.441 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-03-13T21:31:35.545 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-03-13T21:31:35.760 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:35.981 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:52 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:36.048 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:36.436 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:36.436 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:36.436 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-03-13T21:31:36.436 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:36.436 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:31:36.436 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:36.658 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install librados2 2024-03-13T21:31:37.049 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:37.099 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:37.100 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-03-13T21:31:37.100 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:37.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rados 2024-03-13T21:31:37.416 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:58 ago on Wed 13 Mar 2024 09:29:39 PM UTC. 2024-03-13T21:31:37.694 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:37.695 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:37.695 INFO:teuthology.orchestra.run.smithi023.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:37.695 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:37.695 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:31:37.695 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-mirror x86_64 2:15.2.17-0.el8 ceph 2.7 M 2024-03-13T21:31:37.695 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:37.695 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:31:37.695 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:37.695 INFO:teuthology.orchestra.run.smithi023.stdout:Install 1 Package 2024-03-13T21:31:37.695 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:37.696 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 2.7 M 2024-03-13T21:31:37.696 INFO:teuthology.orchestra.run.smithi023.stdout:Installed size: 9.8 M 2024-03-13T21:31:37.696 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:31:37.790 INFO:teuthology.orchestra.run.smithi023.stdout:rbd-mirror-15.2.17-0.el8.x86_64.rpm 29 MB/s | 2.7 MB 00:00 2024-03-13T21:31:37.790 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:37.790 INFO:teuthology.orchestra.run.smithi023.stdout:Total 29 MB/s | 2.7 MB 00:00 2024-03-13T21:31:37.791 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:31:37.803 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:31:37.803 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:31:37.864 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:31:37.868 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:31:37.988 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:41 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:38.224 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:31:38.379 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:38.813 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:38.948 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:38.993 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-03-13T21:31:38.995 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2024-03-13T21:31:38.995 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-03-13T21:31:39.114 DEBUG:teuthology.parallel:result is None 2024-03-13T21:31:39.227 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:39.228 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:39.228 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:31:39.228 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-mirror-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:39.228 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:39.228 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:39.467 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install rbd-nbd 2024-03-13T21:31:39.639 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rados-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:39.688 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:39.690 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-03-13T21:31:39.690 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:39.780 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rgw 2024-03-13T21:31:40.262 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:01:57 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:40.564 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:44 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:41.973 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout:Installing: 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-nbd x86_64 2:15.2.17-0.el8 ceph 140 k 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction Summary 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout:================================================================================ 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout:Install 1 Package 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout:Total download size: 140 k 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout:Installed size: 418 k 2024-03-13T21:31:41.975 INFO:teuthology.orchestra.run.smithi023.stdout:Downloading Packages: 2024-03-13T21:31:42.026 INFO:teuthology.orchestra.run.smithi023.stdout:rbd-nbd-15.2.17-0.el8.x86_64.rpm 2.7 MB/s | 140 kB 00:00 2024-03-13T21:31:42.027 INFO:teuthology.orchestra.run.smithi023.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:42.027 INFO:teuthology.orchestra.run.smithi023.stdout:Total 2.7 MB/s | 140 kB 00:00 2024-03-13T21:31:42.027 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction check 2024-03-13T21:31:42.038 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction check succeeded. 2024-03-13T21:31:42.038 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction test 2024-03-13T21:31:42.087 INFO:teuthology.orchestra.run.smithi023.stdout:Transaction test succeeded. 2024-03-13T21:31:42.090 INFO:teuthology.orchestra.run.smithi023.stdout:Running transaction 2024-03-13T21:31:42.208 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rgw-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:42.210 INFO:teuthology.orchestra.run.smithi023.stdout: Preparing : 1/1 2024-03-13T21:31:42.258 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:42.260 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-03-13T21:31:42.260 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:42.355 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-cephfs 2024-03-13T21:31:42.386 INFO:teuthology.orchestra.run.smithi023.stdout: Installing : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:42.684 INFO:teuthology.orchestra.run.smithi023.stdout: Running scriptlet: rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:43.060 INFO:teuthology.orchestra.run.smithi023.stdout: Verifying : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:43.060 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:43.060 INFO:teuthology.orchestra.run.smithi023.stdout:Installed: 2024-03-13T21:31:43.060 INFO:teuthology.orchestra.run.smithi023.stdout: rbd-nbd-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:43.060 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:31:43.060 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:43.147 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:47 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:43.274 DEBUG:teuthology.orchestra.run.smithi023:> sudo yum -y install librados2 2024-03-13T21:31:44.078 INFO:teuthology.orchestra.run.smithi023.stdout:Last metadata expiration check: 0:02:01 ago on Wed 13 Mar 2024 09:29:43 PM UTC. 2024-03-13T21:31:44.797 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-cephfs-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:44.849 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:44.851 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-03-13T21:31:44.851 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:44.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install python3-rbd 2024-03-13T21:31:45.728 INFO:teuthology.orchestra.run.smithi023.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:45.736 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:49 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:45.780 INFO:teuthology.orchestra.run.smithi023.stdout:Dependencies resolved. 2024-03-13T21:31:45.782 INFO:teuthology.orchestra.run.smithi023.stdout:Nothing to do. 2024-03-13T21:31:45.782 INFO:teuthology.orchestra.run.smithi023.stdout:Complete! 2024-03-13T21:31:45.892 DEBUG:teuthology.parallel:result is None 2024-03-13T21:31:47.385 INFO:teuthology.orchestra.run.smithi064.stdout:Package python3-rbd-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:31:47.435 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:47.437 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-03-13T21:31:47.437 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:47.534 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-fuse 2024-03-13T21:31:48.323 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:52 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:50.035 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:50.036 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:50.036 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:50.036 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:50.036 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:31:50.036 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:15.2.17-0.el8 ceph 73 k 2024-03-13T21:31:50.036 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:50.036 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:31:50.036 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:50.037 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-03-13T21:31:50.037 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:50.037 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 73 k 2024-03-13T21:31:50.037 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 188 k 2024-03-13T21:31:50.037 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:31:50.086 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-fuse-15.2.17-0.el8.x86_64.rpm 1.4 MB/s | 73 kB 00:00 2024-03-13T21:31:50.087 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:50.087 INFO:teuthology.orchestra.run.smithi064.stdout:Total 1.4 MB/s | 73 kB 00:00 2024-03-13T21:31:50.087 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:31:50.098 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:31:50.098 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:31:50.147 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:31:50.150 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:31:50.288 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:31:50.454 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:50.772 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:51.211 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:51.212 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:31:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:51.212 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:51.212 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:51.413 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-mirror 2024-03-13T21:31:52.209 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:56 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:53.911 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:53.912 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:53.912 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:53.912 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:31:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:15.2.17-0.el8 ceph 2.7 M 2024-03-13T21:31:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:53.912 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:31:53.912 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:53.912 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-03-13T21:31:53.912 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:53.913 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 2.7 M 2024-03-13T21:31:53.913 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 9.8 M 2024-03-13T21:31:53.913 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:31:54.031 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-mirror-15.2.17-0.el8.x86_64.rpm 23 MB/s | 2.7 MB 00:00 2024-03-13T21:31:54.031 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:54.032 INFO:teuthology.orchestra.run.smithi064.stdout:Total 23 MB/s | 2.7 MB 00:00 2024-03-13T21:31:54.032 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:31:54.044 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:31:54.044 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:31:54.104 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:31:54.107 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:31:54.469 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:31:54.647 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:55.070 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:55.475 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:55.475 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:55.475 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:31:55.475 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:55.475 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:55.476 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:55.682 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install rbd-nbd 2024-03-13T21:31:56.477 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:00 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:31:58.197 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:31:58.197 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:58.197 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-03-13T21:31:58.198 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:58.198 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-03-13T21:31:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:15.2.17-0.el8 ceph 140 k 2024-03-13T21:31:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:58.198 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-03-13T21:31:58.198 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-03-13T21:31:58.198 INFO:teuthology.orchestra.run.smithi064.stdout:Install 1 Package 2024-03-13T21:31:58.198 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:58.199 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 140 k 2024-03-13T21:31:58.199 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 418 k 2024-03-13T21:31:58.199 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-03-13T21:31:58.233 INFO:teuthology.orchestra.run.smithi064.stdout:rbd-nbd-15.2.17-0.el8.x86_64.rpm 4.0 MB/s | 140 kB 00:00 2024-03-13T21:31:58.234 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-03-13T21:31:58.234 INFO:teuthology.orchestra.run.smithi064.stdout:Total 3.8 MB/s | 140 kB 00:00 2024-03-13T21:31:58.234 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-03-13T21:31:58.245 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-03-13T21:31:58.245 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-03-13T21:31:58.294 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-03-13T21:31:58.298 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-03-13T21:31:58.445 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-03-13T21:31:58.603 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:58.929 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:59.347 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:15.2.17-0.el8.x86_64 1/1 2024-03-13T21:31:59.347 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:59.347 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-03-13T21:31:59.347 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:15.2.17-0.el8.x86_64 2024-03-13T21:31:59.347 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:31:59.347 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:31:59.546 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install librados2 2024-03-13T21:32:00.339 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:04 ago on Wed 13 Mar 2024 09:29:56 PM UTC. 2024-03-13T21:32:02.005 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:15.2.17-0.el8.x86_64 is already installed. 2024-03-13T21:32:02.055 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-03-13T21:32:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:Nothing to do. 2024-03-13T21:32:02.056 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-03-13T21:32:02.183 DEBUG:teuthology.parallel:result is None 2024-03-13T21:32:02.183 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T21:32:02.183 INFO:teuthology.packaging:ref: None 2024-03-13T21:32:02.183 INFO:teuthology.packaging:tag: None 2024-03-13T21:32:02.183 INFO:teuthology.packaging:branch: octopus 2024-03-13T21:32:02.183 INFO:teuthology.packaging:sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 2024-03-13T21:32:02.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2024-03-13T21:32:02.617 DEBUG:teuthology.orchestra.run.smithi023:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-13T21:32:02.645 INFO:teuthology.orchestra.run.smithi023.stdout:15.2.17-0.el8 2024-03-13T21:32:02.645 INFO:teuthology.packaging:The installed version of ceph is 15.2.17-0.el8 2024-03-13T21:32:02.646 INFO:teuthology.task.install:The correct ceph version 15.2.17-0 is installed. 2024-03-13T21:32:02.648 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T21:32:02.648 INFO:teuthology.packaging:ref: None 2024-03-13T21:32:02.648 INFO:teuthology.packaging:tag: None 2024-03-13T21:32:02.648 INFO:teuthology.packaging:branch: octopus 2024-03-13T21:32:02.648 INFO:teuthology.packaging:sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 2024-03-13T21:32:02.648 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2024-03-13T21:32:02.860 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-13T21:32:02.887 INFO:teuthology.orchestra.run.smithi064.stdout:15.2.17-0.el8 2024-03-13T21:32:02.888 INFO:teuthology.packaging:The installed version of ceph is 15.2.17-0.el8 2024-03-13T21:32:02.888 INFO:teuthology.task.install:The correct ceph version 15.2.17-0 is installed. 2024-03-13T21:32:02.889 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T21:32:02.890 INFO:teuthology.packaging:ref: None 2024-03-13T21:32:02.890 INFO:teuthology.packaging:tag: None 2024-03-13T21:32:02.890 INFO:teuthology.packaging:branch: octopus 2024-03-13T21:32:02.890 INFO:teuthology.packaging:sha1: d16d1766eb379253122a6acda1cdd17a0b9c3aa3 2024-03-13T21:32:02.890 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=octopus 2024-03-13T21:32:03.102 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-13T21:32:03.129 INFO:teuthology.orchestra.run.smithi077.stdout:15.2.17-0.el8 2024-03-13T21:32:03.129 INFO:teuthology.packaging:The installed version of ceph is 15.2.17-0.el8 2024-03-13T21:32:03.129 INFO:teuthology.task.install:The correct ceph version 15.2.17-0 is installed. 2024-03-13T21:32:03.131 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-13T21:32:03.131 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:03.131 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-13T21:32:03.165 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-13T21:32:03.166 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-13T21:32:03.199 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-03-13T21:32:03.199 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-13T21:32:03.231 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-13T21:32:03.231 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:03.231 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/usr/bin/daemon-helper 2024-03-13T21:32:03.264 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-13T21:32:03.334 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-13T21:32:03.334 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-03-13T21:32:03.367 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-13T21:32:03.438 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-03-13T21:32:03.438 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2024-03-13T21:32:03.470 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-13T21:32:03.541 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-13T21:32:03.541 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:03.541 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-13T21:32:03.572 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-13T21:32:03.642 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-13T21:32:03.642 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-13T21:32:03.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-13T21:32:03.747 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-03-13T21:32:03.747 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-13T21:32:03.779 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-13T21:32:03.849 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-13T21:32:03.849 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:03.850 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/usr/bin/stdin-killer 2024-03-13T21:32:03.879 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-13T21:32:03.949 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-13T21:32:03.950 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-03-13T21:32:03.983 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-13T21:32:04.055 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-03-13T21:32:04.055 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2024-03-13T21:32:04.087 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-13T21:32:04.157 INFO:teuthology.run_tasks:Running task print... 2024-03-13T21:32:04.167 INFO:teuthology.task.print:**** done installing octopus 2024-03-13T21:32:04.167 INFO:teuthology.run_tasks:Running task ceph... 2024-03-13T21:32:04.275 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-13T21:32:04.275 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod 777 /var/log/ceph 2024-03-13T21:32:04.278 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /var/log/ceph 2024-03-13T21:32:04.280 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /var/log/ceph 2024-03-13T21:32:04.311 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-13T21:32:04.311 DEBUG:teuthology.orchestra.run.smithi023:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-13T21:32:04.351 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-13T21:32:04.354 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-13T21:32:04.384 INFO:tasks.ceph:Creating extra log directories... 2024-03-13T21:32:04.385 DEBUG:teuthology.orchestra.run.smithi023:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-13T21:32:04.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-13T21:32:04.427 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-13T21:32:04.466 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-13T21:32:04.466 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request', 'missing required features'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-13T21:32:04.467 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-03-13_20:49:58-fs-wip-batrick-testing-20240310.014948-distro-default-smithi/7599086', 'branch': 'wip-batrick-testing-20240310.014948', 'description': 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7599086', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'install ceph/octopus latest\n'}], 'name': 'pdonnell-2024-03-13_20:49:58-fs-wip-batrick-testing-20240310.014948-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20240310.014948'}, 'ceph': {'cephfs': {'max_mds': 2}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request', 'missing required features'], 'sha1': 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20240310.014948', 'sha1': '32d4e26ee101db040501914293e54a3c24a95078'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'mds.b', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['client.0'], ['client.1']], 'seed': 6353, 'sha1': 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3', 'sleep_before_teardown': 0, 'subset': '383/512', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20240310.014948', 'suite_path': '/home/teuthworker/src/github.com_batrick_ceph_32d4e26ee101db040501914293e54a3c24a95078/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/batrick/ceph', 'suite_sha1': '32d4e26ee101db040501914293e54a3c24a95078', 'targets': {'smithi023.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB92a+610flmdc9mTv7MjZOowm+0vqk8lSjlxOwWrTCZsijMLF8mQb+hW0QQADandrsYZh/uf/A3xqGpgw5A1MM=', 'smithi064.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJAls2lVBYvbLoW7nd1zLzZh5uWEuXvqhVopnJeP/7VnUJ8Q03mEv57vINDMw7/mDVEH/jmbsV8HNVp5WiMCAno=', 'smithi077.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAi4AMR6ELlSYd3eg2D4jN5YdYwyw4pbrt10cH2HUPI8+eTzjBwLadd5luuvZ3NIUXP1i6idunfDt7CUMwQ8Drk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'octopus', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3'}}, {'print': '**** done installing octopus'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request', 'missing required features'], 'cephfs': {'max_mds': 2}, 'flavor': 'default', 'fs': 'xfs', 'sha1': 'd16d1766eb379253122a6acda1cdd17a0b9c3aa3', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client octopus']}}, {'print': '**** done ceph'}, {'ceph-fuse': None}, {'print': '**** done octopus client'}, {'workunit': {'clients': {'all': ['suites/fsstress.sh']}}}, {'print': '**** done fsstress'}, {'mds_pre_upgrade': None}, {'print': '**** done mds pre-upgrade sequence'}, {'install.upgrade': {'mon.a': {'branch': 'quincy'}}}, {'print': '**** done install.upgrade the host'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'ceph.stop': ['mds.*']}, {'ceph.restart': {'daemons': ['mds.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph osd dump -f json-pretty', 'ceph versions', 'ceph osd require-osd-release quincy', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}, {'exec': {'mon.a': ['ceph fs dump --format=json-pretty', 'ceph fs required_client_features cephfs add metric_collect']}}, {'sleep': {'duration': 5}}, {'fs.clients_evicted': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-13_20:49:58', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-13T21:32:04.467 DEBUG:teuthology.orchestra.run.smithi023:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-13T21:32:04.494 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-13T21:32:04.499 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-13T21:32:04.518 DEBUG:teuthology.orchestra.run.smithi023:> sudo install -d -m0777 -- /var/run/ceph 2024-03-13T21:32:04.554 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m0777 -- /var/run/ceph 2024-03-13T21:32:04.559 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m0777 -- /var/run/ceph 2024-03-13T21:32:04.588 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:04.588 DEBUG:teuthology.orchestra.run.smithi023:> dd if=/scratch_devs of=/dev/stdout 2024-03-13T21:32:04.642 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-13T21:32:04.642 DEBUG:teuthology.orchestra.run.smithi023:> stat /dev/vg_nvme/lv_1 2024-03-13T21:32:04.701 INFO:teuthology.orchestra.run.smithi023.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-13T21:32:04.702 INFO:teuthology.orchestra.run.smithi023.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T21:32:04.702 INFO:teuthology.orchestra.run.smithi023.stdout:Device: 6h/6d Inode: 168336 Links: 1 2024-03-13T21:32:04.702 INFO:teuthology.orchestra.run.smithi023.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T21:32:04.702 INFO:teuthology.orchestra.run.smithi023.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T21:32:04.702 INFO:teuthology.orchestra.run.smithi023.stdout:Access: 2024-03-13 21:31:38.565177023 +0000 2024-03-13T21:32:04.702 INFO:teuthology.orchestra.run.smithi023.stdout:Modify: 2024-03-13 21:28:50.685657109 +0000 2024-03-13T21:32:04.702 INFO:teuthology.orchestra.run.smithi023.stdout:Change: 2024-03-13 21:28:50.685657109 +0000 2024-03-13T21:32:04.702 INFO:teuthology.orchestra.run.smithi023.stdout: Birth: - 2024-03-13T21:32:04.702 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-13T21:32:04.770 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records in 2024-03-13T21:32:04.770 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records out 2024-03-13T21:32:04.770 INFO:teuthology.orchestra.run.smithi023.stderr:512 bytes copied, 0.000448997 s, 1.1 MB/s 2024-03-13T21:32:04.771 DEBUG:teuthology.orchestra.run.smithi023:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-13T21:32:04.832 DEBUG:teuthology.orchestra.run.smithi023:> stat /dev/vg_nvme/lv_2 2024-03-13T21:32:04.891 INFO:teuthology.orchestra.run.smithi023.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-13T21:32:04.891 INFO:teuthology.orchestra.run.smithi023.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T21:32:04.891 INFO:teuthology.orchestra.run.smithi023.stdout:Device: 6h/6d Inode: 168443 Links: 1 2024-03-13T21:32:04.891 INFO:teuthology.orchestra.run.smithi023.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T21:32:04.891 INFO:teuthology.orchestra.run.smithi023.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T21:32:04.892 INFO:teuthology.orchestra.run.smithi023.stdout:Access: 2024-03-13 21:31:38.565177023 +0000 2024-03-13T21:32:04.892 INFO:teuthology.orchestra.run.smithi023.stdout:Modify: 2024-03-13 21:28:51.026654617 +0000 2024-03-13T21:32:04.892 INFO:teuthology.orchestra.run.smithi023.stdout:Change: 2024-03-13 21:28:51.026654617 +0000 2024-03-13T21:32:04.892 INFO:teuthology.orchestra.run.smithi023.stdout: Birth: - 2024-03-13T21:32:04.892 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-13T21:32:04.959 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records in 2024-03-13T21:32:04.959 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records out 2024-03-13T21:32:04.960 INFO:teuthology.orchestra.run.smithi023.stderr:512 bytes copied, 0.000287399 s, 1.8 MB/s 2024-03-13T21:32:04.961 DEBUG:teuthology.orchestra.run.smithi023:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-13T21:32:05.021 DEBUG:teuthology.orchestra.run.smithi023:> stat /dev/vg_nvme/lv_3 2024-03-13T21:32:05.080 INFO:teuthology.orchestra.run.smithi023.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-13T21:32:05.080 INFO:teuthology.orchestra.run.smithi023.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T21:32:05.080 INFO:teuthology.orchestra.run.smithi023.stdout:Device: 6h/6d Inode: 169151 Links: 1 2024-03-13T21:32:05.080 INFO:teuthology.orchestra.run.smithi023.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T21:32:05.080 INFO:teuthology.orchestra.run.smithi023.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T21:32:05.080 INFO:teuthology.orchestra.run.smithi023.stdout:Access: 2024-03-13 21:31:38.565177023 +0000 2024-03-13T21:32:05.080 INFO:teuthology.orchestra.run.smithi023.stdout:Modify: 2024-03-13 21:28:51.368652119 +0000 2024-03-13T21:32:05.080 INFO:teuthology.orchestra.run.smithi023.stdout:Change: 2024-03-13 21:28:51.368652119 +0000 2024-03-13T21:32:05.080 INFO:teuthology.orchestra.run.smithi023.stdout: Birth: - 2024-03-13T21:32:05.080 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-13T21:32:05.150 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records in 2024-03-13T21:32:05.150 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records out 2024-03-13T21:32:05.150 INFO:teuthology.orchestra.run.smithi023.stderr:512 bytes copied, 0.000421251 s, 1.2 MB/s 2024-03-13T21:32:05.151 DEBUG:teuthology.orchestra.run.smithi023:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-13T21:32:05.210 DEBUG:teuthology.orchestra.run.smithi023:> stat /dev/vg_nvme/lv_4 2024-03-13T21:32:05.269 INFO:teuthology.orchestra.run.smithi023.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-13T21:32:05.269 INFO:teuthology.orchestra.run.smithi023.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T21:32:05.269 INFO:teuthology.orchestra.run.smithi023.stdout:Device: 6h/6d Inode: 169258 Links: 1 2024-03-13T21:32:05.270 INFO:teuthology.orchestra.run.smithi023.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T21:32:05.270 INFO:teuthology.orchestra.run.smithi023.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T21:32:05.270 INFO:teuthology.orchestra.run.smithi023.stdout:Access: 2024-03-13 21:31:38.566177015 +0000 2024-03-13T21:32:05.270 INFO:teuthology.orchestra.run.smithi023.stdout:Modify: 2024-03-13 21:28:51.726649504 +0000 2024-03-13T21:32:05.270 INFO:teuthology.orchestra.run.smithi023.stdout:Change: 2024-03-13 21:28:51.726649504 +0000 2024-03-13T21:32:05.270 INFO:teuthology.orchestra.run.smithi023.stdout: Birth: - 2024-03-13T21:32:05.270 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-13T21:32:05.336 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records in 2024-03-13T21:32:05.336 INFO:teuthology.orchestra.run.smithi023.stderr:1+0 records out 2024-03-13T21:32:05.336 INFO:teuthology.orchestra.run.smithi023.stderr:512 bytes copied, 0.00027123 s, 1.9 MB/s 2024-03-13T21:32:05.337 DEBUG:teuthology.orchestra.run.smithi023:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-13T21:32:05.397 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-13T21:32:05.397 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi023.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-03-13T21:32:05.397 INFO:tasks.ceph:Generating config... 2024-03-13T21:32:05.398 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-03-13T21:32:05.398 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-03-13T21:32:05.398 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2024-03-13T21:32:05.398 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2024-03-13T21:32:05.398 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2024-03-13T21:32:05.399 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-13T21:32:05.399 INFO:tasks.ceph:[client] debug client = 20 2024-03-13T21:32:05.399 INFO:tasks.ceph:[client] debug ms = 1 2024-03-13T21:32:05.399 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-13T21:32:05.399 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-13T21:32:05.399 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-13T21:32:05.400 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-13T21:32:05.400 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-13T21:32:05.400 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-13T21:32:05.400 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-13T21:32:05.400 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-13T21:32:05.400 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-13T21:32:05.401 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-13T21:32:05.401 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-13T21:32:05.401 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-13T21:32:05.401 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-13T21:32:05.401 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-13T21:32:05.401 INFO:tasks.ceph:Setting up mon.a... 2024-03-13T21:32:05.401 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-13T21:32:05.553 INFO:teuthology.orchestra.run.smithi023.stdout:creating /etc/ceph/ceph.keyring 2024-03-13T21:32:05.558 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-13T21:32:05.599 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-13T21:32:05.665 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.23'), ('mon.b', '[v2:172.21.15.23:3301,v1:172.21.15.23:6790]'), ('mon.c', '[v2:172.21.15.23:3302,v1:172.21.15.23:6791]')] 2024-03-13T21:32:05.666 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.23,[v2:172.21.15.23:3301,v1:172.21.15.23:6790],[v2:172.21.15.23:3302,v1:172.21.15.23:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds.c': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-13T21:32:05.667 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:05.667 DEBUG:teuthology.orchestra.run.smithi023:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-13T21:32:05.724 DEBUG:teuthology.orchestra.run.smithi023:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.23 --addv b '[v2:172.21.15.23:3301,v1:172.21.15.23:6790]' --addv c '[v2:172.21.15.23:3302,v1:172.21.15.23:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-13T21:32:05.808 INFO:teuthology.orchestra.run.smithi023.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-13T21:32:05.808 INFO:teuthology.orchestra.run.smithi023.stdout:monmaptool: generated fsid 19813cba-5ddc-4ec3-85b1-5e37baeea636 2024-03-13T21:32:05.808 INFO:teuthology.orchestra.run.smithi023.stdout:epoch 0 2024-03-13T21:32:05.808 INFO:teuthology.orchestra.run.smithi023.stdout:fsid 19813cba-5ddc-4ec3-85b1-5e37baeea636 2024-03-13T21:32:05.808 INFO:teuthology.orchestra.run.smithi023.stdout:last_changed 2024-03-13T21:32:05.818569+0000 2024-03-13T21:32:05.808 INFO:teuthology.orchestra.run.smithi023.stdout:created 2024-03-13T21:32:05.818569+0000 2024-03-13T21:32:05.809 INFO:teuthology.orchestra.run.smithi023.stdout:min_mon_release 0 (unknown) 2024-03-13T21:32:05.809 INFO:teuthology.orchestra.run.smithi023.stdout:0: [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] mon.a 2024-03-13T21:32:05.809 INFO:teuthology.orchestra.run.smithi023.stdout:1: [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] mon.b 2024-03-13T21:32:05.809 INFO:teuthology.orchestra.run.smithi023.stdout:2: [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] mon.c 2024-03-13T21:32:05.809 INFO:teuthology.orchestra.run.smithi023.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-13T21:32:05.810 DEBUG:teuthology.orchestra.run.smithi023:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-13T21:32:05.827 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 19813cba-5ddc-4ec3-85b1-5e37baeea636... 2024-03-13T21:32:05.828 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-13T21:32:05.870 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-13T21:32:05.872 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout:[global] 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: chdir = "" 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: auth supported = cephx 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: filestore xattr use omap = true 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: mon clock drift allowed = 1.000 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: osd crush chooseleaf type = 0 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: auth debug = true 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: ms die on old message = true 2024-03-13T21:32:05.914 INFO:teuthology.orchestra.run.smithi023.stdout: ms die on bug = true 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: mon pg warn max object skew = 0 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: # disable pg_autoscaler by default for new pools 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: osd pool default size = 2 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd allow primary affinity = true 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd allow pg remap = true 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on legacy crush tunables = false 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on crush straw calc version zero = false 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on no sortbitwise = false 2024-03-13T21:32:05.915 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on osd down out interval zero = false 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on too few osds = false 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: mon_allow_pool_size_one = true 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: osd default data pool replay window = 5 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: mon allow pool delete = true 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: mon cluster log file level = debug 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: debug asserts on shutdown = true 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: mon health detail to clog = false 2024-03-13T21:32:05.916 INFO:teuthology.orchestra.run.smithi023.stdout: mon host = "172.21.15.23,[v2:172.21.15.23:3301,v1:172.21.15.23:6790],[v2:172.21.15.23:3302,v1:172.21.15.23:6791]" 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on pool no app = False 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: ms bind msgr2 = False 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: bluestore warn on legacy statfs = False 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: bluestore warn on no per pool omap = False 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: mon pg warn min per osd = 0 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: fsid = 19813cba-5ddc-4ec3-85b1-5e37baeea636 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout:[osd] 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: osd journal size = 100 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: osd scrub load threshold = 5.0 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: osd scrub max interval = 600 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: osd mclock profile = high_recovery_ops 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: osd recover clone overlap = true 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: osd recovery max chunk = 1048576 2024-03-13T21:32:05.917 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug shutdown = true 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug op order = true 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug verify stray on activate = true 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug trim objects = true 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: osd open classes on start = true 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug pg log writeout = true 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: osd deep scrub update digest min age = 30 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: osd map max advance = 10 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: journal zero on create = true 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.918 INFO:teuthology.orchestra.run.smithi023.stdout: filestore ondisk finisher threads = 3 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: filestore apply finisher threads = 3 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: bdev debug aio = true 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: osd debug misdirected ops = true 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: bdev async discard = True 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: bdev enable discard = True 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: bluestore allocator = bitmap 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: bluestore block size = 96636764160 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: bluestore fsck on mount = True 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: debug bluefs = 1/20 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: debug bluestore = 1/20 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: debug ms = 1 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: debug osd = 20 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: debug rocksdb = 4/10 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd backfillfull_ratio = 0.85 2024-03-13T21:32:05.919 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd full ratio = 0.9 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd nearfull ratio = 0.8 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: osd failsafe full ratio = 0.95 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: osd objectstore = bluestore 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: osd op complaint time = 180 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout:[mgr] 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: debug ms = 1 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: debug mgr = 20 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: debug mon = 20 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: debug auth = 20 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: mon reweight min pgs per osd = 4 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: mon reweight min bytes per osd = 10 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: mgr/telemetry/nag = false 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout:[mon] 2024-03-13T21:32:05.920 INFO:teuthology.orchestra.run.smithi023.stdout: debug ms = 1 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: debug mon = 20 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: debug paxos = 20 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: debug auth = 20 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: mon data avail warn = 5 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: mon mgr mkfs grace = 240 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: mon reweight min pgs per osd = 4 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd reporter subtree level = osd 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: mon osd prime pg temp = true 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: mon reweight min bytes per osd = 10 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: auth mon ticket ttl = 660 # 11m 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: auth service ticket ttl = 240 # 4m 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: # don't complain about insecure global_id in the test suite 2024-03-13T21:32:05.921 INFO:teuthology.orchestra.run.smithi023.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: # 1m isn't quite enough 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: mon_down_mkfs_grace = 2m 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: mon_warn_on_filestore_osds = false 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: mon op complaint time = 120 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: mon warn on osd down out interval zero = False 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout:[client] 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: rgw cache enabled = true 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: rgw enable ops log = true 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: rgw enable usage log = true 2024-03-13T21:32:05.922 INFO:teuthology.orchestra.run.smithi023.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout: client mount timeout = 600 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout: debug client = 20 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout: debug ms = 1 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout: rados mon op timeout = 900 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout: rados osd op timeout = 900 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout:[mon.a] 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout:[mon.b] 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout:[mon.c] 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.a] 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.b] 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.c] 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout:[mds] 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout: debug mds = 20 2024-03-13T21:32:05.923 INFO:teuthology.orchestra.run.smithi023.stdout: debug mds balancer = 20 2024-03-13T21:32:05.924 INFO:teuthology.orchestra.run.smithi023.stdout: debug ms = 1 2024-03-13T21:32:05.924 INFO:teuthology.orchestra.run.smithi023.stdout: mds debug frag = True 2024-03-13T21:32:05.924 INFO:teuthology.orchestra.run.smithi023.stdout: mds debug scatterstat = True 2024-03-13T21:32:05.924 INFO:teuthology.orchestra.run.smithi023.stdout: mds op complaint time = 180 2024-03-13T21:32:05.924 INFO:teuthology.orchestra.run.smithi023.stdout: mds verify scatter = True 2024-03-13T21:32:05.924 INFO:teuthology.orchestra.run.smithi023.stdout: osd op complaint time = 180 2024-03-13T21:32:05.924 INFO:teuthology.orchestra.run.smithi023.stdout: rados mon op timeout = 900 2024-03-13T21:32:05.924 INFO:teuthology.orchestra.run.smithi023.stdout: rados osd op timeout = 900 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout:[global] 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: chdir = "" 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: auth supported = cephx 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: filestore xattr use omap = true 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: mon clock drift allowed = 1.000 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: osd crush chooseleaf type = 0 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: auth debug = true 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.925 INFO:teuthology.orchestra.run.smithi077.stdout: ms die on old message = true 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: ms die on bug = true 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: mon pg warn max object skew = 0 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: # disable pg_autoscaler by default for new pools 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: osd pool default size = 2 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd allow primary affinity = true 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd allow pg remap = true 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on legacy crush tunables = false 2024-03-13T21:32:05.926 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on crush straw calc version zero = false 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on no sortbitwise = false 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on osd down out interval zero = false 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on too few osds = false 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: mon_allow_pool_size_one = true 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: osd default data pool replay window = 5 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: mon allow pool delete = true 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: mon cluster log file level = debug 2024-03-13T21:32:05.927 INFO:teuthology.orchestra.run.smithi077.stdout: debug asserts on shutdown = true 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: mon health detail to clog = false 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: mon host = "172.21.15.23,[v2:172.21.15.23:3301,v1:172.21.15.23:6790],[v2:172.21.15.23:3302,v1:172.21.15.23:6791]" 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on pool no app = False 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: ms bind msgr2 = False 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore warn on legacy statfs = False 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore warn on no per pool omap = False 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: mon pg warn min per osd = 0 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: fsid = 19813cba-5ddc-4ec3-85b1-5e37baeea636 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout:[osd] 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: osd journal size = 100 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub load threshold = 5.0 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: osd scrub max interval = 600 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: osd mclock profile = high_recovery_ops 2024-03-13T21:32:05.928 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: osd recover clone overlap = true 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: osd recovery max chunk = 1048576 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug shutdown = true 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug op order = true 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug verify stray on activate = true 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug trim objects = true 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: osd open classes on start = true 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug pg log writeout = true 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: osd deep scrub update digest min age = 30 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.929 INFO:teuthology.orchestra.run.smithi077.stdout: osd map max advance = 10 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: journal zero on create = true 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: filestore ondisk finisher threads = 3 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: filestore apply finisher threads = 3 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: bdev debug aio = true 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: osd debug misdirected ops = true 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: bdev async discard = True 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: bdev enable discard = True 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore allocator = bitmap 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore block size = 96636764160 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: bluestore fsck on mount = True 2024-03-13T21:32:05.930 INFO:teuthology.orchestra.run.smithi077.stdout: debug bluefs = 1/20 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: debug bluestore = 1/20 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: debug osd = 20 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: debug rocksdb = 4/10 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd backfillfull_ratio = 0.85 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd full ratio = 0.9 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd nearfull ratio = 0.8 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: osd failsafe full ratio = 0.95 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: osd objectstore = bluestore 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: osd op complaint time = 180 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout:[mgr] 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2024-03-13T21:32:05.931 INFO:teuthology.orchestra.run.smithi077.stdout: debug mgr = 20 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: debug mon = 20 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: debug auth = 20 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min pgs per osd = 4 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min bytes per osd = 10 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: mgr/telemetry/nag = false 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout:[mon] 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: debug mon = 20 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: debug paxos = 20 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: debug auth = 20 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: mon data avail warn = 5 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: mon mgr mkfs grace = 240 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min pgs per osd = 4 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd reporter subtree level = osd 2024-03-13T21:32:05.932 INFO:teuthology.orchestra.run.smithi077.stdout: mon osd prime pg temp = true 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: mon reweight min bytes per osd = 10 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: auth mon ticket ttl = 660 # 11m 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: auth service ticket ttl = 240 # 4m 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: # don't complain about insecure global_id in the test suite 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: # 1m isn't quite enough 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: mon_down_mkfs_grace = 2m 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: mon_warn_on_filestore_osds = false 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: mon op complaint time = 120 2024-03-13T21:32:05.933 INFO:teuthology.orchestra.run.smithi077.stdout: mon warn on osd down out interval zero = False 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout:[client] 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: rgw cache enabled = true 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: rgw enable ops log = true 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: rgw enable usage log = true 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: client mount timeout = 600 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: debug client = 20 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: rados mon op timeout = 900 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout: rados osd op timeout = 900 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.a] 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.b] 2024-03-13T21:32:05.934 INFO:teuthology.orchestra.run.smithi077.stdout:[mon.c] 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.a] 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.b] 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.c] 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout:[mds] 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout: debug mds = 20 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout: debug mds balancer = 20 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout: debug ms = 1 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout: mds debug frag = True 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout: mds debug scatterstat = True 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout: mds op complaint time = 180 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout: mds verify scatter = True 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout: osd op complaint time = 180 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout: rados mon op timeout = 900 2024-03-13T21:32:05.935 INFO:teuthology.orchestra.run.smithi077.stdout: rados osd op timeout = 900 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: chdir = "" 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: auth supported = cephx 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: filestore xattr use omap = true 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: mon clock drift allowed = 1.000 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: osd crush chooseleaf type = 0 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: auth debug = true 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on old message = true 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: ms die on bug = true 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.936 INFO:teuthology.orchestra.run.smithi064.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn max object skew = 0 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: # disable pg_autoscaler by default for new pools 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default size = 2 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow primary affinity = true 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd allow pg remap = true 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on legacy crush tunables = false 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on crush straw calc version zero = false 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on no sortbitwise = false 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = false 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on too few osds = false 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-13T21:32:05.937 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: mon_allow_pool_size_one = true 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: osd default data pool replay window = 5 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: mon allow pool delete = true 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: mon cluster log file level = debug 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: debug asserts on shutdown = true 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: mon health detail to clog = false 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: mon host = "172.21.15.23,[v2:172.21.15.23:3301,v1:172.21.15.23:6790],[v2:172.21.15.23:3302,v1:172.21.15.23:6791]" 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on pool no app = False 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: ms bind msgr2 = False 2024-03-13T21:32:05.938 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore warn on legacy statfs = False 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore warn on no per pool omap = False 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: mon pg warn min per osd = 0 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = 19813cba-5ddc-4ec3-85b1-5e37baeea636 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout:[osd] 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: osd journal size = 100 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub load threshold = 5.0 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: osd scrub max interval = 600 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: osd mclock profile = high_recovery_ops 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: osd recover clone overlap = true 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: osd recovery max chunk = 1048576 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.939 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug shutdown = true 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug op order = true 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug verify stray on activate = true 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug trim objects = true 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: osd open classes on start = true 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug pg log writeout = true 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: osd deep scrub update digest min age = 30 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: osd map max advance = 10 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: journal zero on create = true 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: filestore ondisk finisher threads = 3 2024-03-13T21:32:05.940 INFO:teuthology.orchestra.run.smithi064.stdout: filestore apply finisher threads = 3 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: bdev debug aio = true 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: osd debug misdirected ops = true 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: bdev async discard = True 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: bdev enable discard = True 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore allocator = bitmap 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore block size = 96636764160 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: bluestore fsck on mount = True 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluefs = 1/20 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: debug bluestore = 1/20 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: debug osd = 20 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: debug rocksdb = 4/10 2024-03-13T21:32:05.941 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd backfillfull_ratio = 0.85 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd full ratio = 0.9 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd nearfull ratio = 0.8 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: osd failsafe full ratio = 0.95 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: osd objectstore = bluestore 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout:[mgr] 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: debug mgr = 20 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: mgr/telemetry/nag = false 2024-03-13T21:32:05.942 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout:[mon] 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: debug mon = 20 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: debug paxos = 20 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: debug auth = 20 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: mon data avail warn = 5 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: mon mgr mkfs grace = 240 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min pgs per osd = 4 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd reporter subtree level = osd 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: mon osd prime pg temp = true 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: mon reweight min bytes per osd = 10 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: auth mon ticket ttl = 660 # 11m 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: auth service ticket ttl = 240 # 4m 2024-03-13T21:32:05.943 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: # don't complain about insecure global_id in the test suite 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: # 1m isn't quite enough 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: mon_down_mkfs_grace = 2m 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: mon_warn_on_filestore_osds = false 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: mon op complaint time = 120 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: mon warn on osd down out interval zero = False 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout:[client] 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: rgw cache enabled = true 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable ops log = true 2024-03-13T21:32:05.944 INFO:teuthology.orchestra.run.smithi064.stdout: rgw enable usage log = true 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout: client mount timeout = 600 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout: debug client = 20 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.a] 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.b] 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout:[mon.c] 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a] 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b] 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c] 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout:[mds] 2024-03-13T21:32:05.945 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds = 20 2024-03-13T21:32:05.946 INFO:teuthology.orchestra.run.smithi064.stdout: debug mds balancer = 20 2024-03-13T21:32:05.946 INFO:teuthology.orchestra.run.smithi064.stdout: debug ms = 1 2024-03-13T21:32:05.946 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug frag = True 2024-03-13T21:32:05.946 INFO:teuthology.orchestra.run.smithi064.stdout: mds debug scatterstat = True 2024-03-13T21:32:05.946 INFO:teuthology.orchestra.run.smithi064.stdout: mds op complaint time = 180 2024-03-13T21:32:05.946 INFO:teuthology.orchestra.run.smithi064.stdout: mds verify scatter = True 2024-03-13T21:32:05.946 INFO:teuthology.orchestra.run.smithi064.stdout: osd op complaint time = 180 2024-03-13T21:32:05.946 INFO:teuthology.orchestra.run.smithi064.stdout: rados mon op timeout = 900 2024-03-13T21:32:05.946 INFO:teuthology.orchestra.run.smithi064.stdout: rados osd op timeout = 900 2024-03-13T21:32:05.948 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-13T21:32:05.948 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-13T21:32:06.040 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-13T21:32:06.040 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:06.040 DEBUG:teuthology.orchestra.run.smithi023:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-13T21:32:06.057 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:06.057 DEBUG:teuthology.orchestra.run.smithi023:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-13T21:32:06.115 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi023.front.sepia.ceph.com 2024-03-13T21:32:06.115 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:06.116 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-13T21:32:06.116 DEBUG:teuthology.orchestra.run.smithi023:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-13T21:32:06.192 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:06.193 DEBUG:teuthology.orchestra.run.smithi023:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-13T21:32:06.250 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi064.front.sepia.ceph.com 2024-03-13T21:32:06.251 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-13T21:32:06.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-13T21:32:06.251 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-13T21:32:06.295 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-13T21:32:06.295 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-13T21:32:06.355 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi077.front.sepia.ceph.com 2024-03-13T21:32:06.355 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-03-13T21:32:06.355 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-13T21:32:06.356 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-13T21:32:06.399 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-03-13T21:32:06.399 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-13T21:32:06.459 INFO:tasks.ceph:Setting up mon nodes... 2024-03-13T21:32:06.459 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-13T21:32:06.459 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-13T21:32:06.509 INFO:teuthology.orchestra.run.smithi023.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-13T21:32:06.511 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-13T21:32:06.561 INFO:teuthology.orchestra.run.smithi023.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-13T21:32:06.563 INFO:tasks.ceph:Setting up mds nodes... 2024-03-13T21:32:06.564 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-13T21:32:06.613 INFO:teuthology.orchestra.run.smithi023.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-13T21:32:06.615 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-13T21:32:06.642 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-13T21:32:06.732 INFO:teuthology.orchestra.run.smithi023.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-13T21:32:06.734 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-13T21:32:06.762 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-13T21:32:06.852 INFO:teuthology.orchestra.run.smithi023.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-13T21:32:06.854 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-13T21:32:06.881 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-13T21:32:06.881 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-13T21:32:07.052 INFO:teuthology.orchestra.run.smithi064.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-13T21:32:07.067 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-13T21:32:07.185 INFO:teuthology.orchestra.run.smithi077.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-13T21:32:07.199 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-13T21:32:07.199 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi023.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-03-13T21:32:07.199 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-13T21:32:07.226 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-13T21:32:07.227 INFO:tasks.ceph:role: osd.0 2024-03-13T21:32:07.227 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi023.front.sepia.ceph.com 2024-03-13T21:32:07.227 DEBUG:teuthology.orchestra.run.smithi023:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-13T21:32:07.321 INFO:teuthology.orchestra.run.smithi023.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T21:32:07.322 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T21:32:07.322 INFO:teuthology.orchestra.run.smithi023.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T21:32:07.322 INFO:teuthology.orchestra.run.smithi023.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-13T21:32:07.322 INFO:teuthology.orchestra.run.smithi023.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T21:32:07.322 INFO:teuthology.orchestra.run.smithi023.stdout: = sunit=0 swidth=0 blks 2024-03-13T21:32:07.322 INFO:teuthology.orchestra.run.smithi023.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T21:32:07.322 INFO:teuthology.orchestra.run.smithi023.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-13T21:32:07.322 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T21:32:07.322 INFO:teuthology.orchestra.run.smithi023.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T21:32:08.604 INFO:teuthology.orchestra.run.smithi023.stdout:Discarding blocks...Done. 2024-03-13T21:32:08.607 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi023.front.sepia.ceph.com -o noatime 2024-03-13T21:32:08.607 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-13T21:32:08.655 DEBUG:teuthology.orchestra.run.smithi023:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-13T21:32:08.690 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-13T21:32:08.760 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-13T21:32:08.761 INFO:tasks.ceph:role: osd.1 2024-03-13T21:32:08.761 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi023.front.sepia.ceph.com 2024-03-13T21:32:08.761 DEBUG:teuthology.orchestra.run.smithi023:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-13T21:32:08.857 INFO:teuthology.orchestra.run.smithi023.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T21:32:08.857 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T21:32:08.857 INFO:teuthology.orchestra.run.smithi023.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T21:32:08.858 INFO:teuthology.orchestra.run.smithi023.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-13T21:32:08.858 INFO:teuthology.orchestra.run.smithi023.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T21:32:08.858 INFO:teuthology.orchestra.run.smithi023.stdout: = sunit=0 swidth=0 blks 2024-03-13T21:32:08.858 INFO:teuthology.orchestra.run.smithi023.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T21:32:08.858 INFO:teuthology.orchestra.run.smithi023.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-13T21:32:08.858 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T21:32:08.858 INFO:teuthology.orchestra.run.smithi023.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T21:32:10.142 INFO:teuthology.orchestra.run.smithi023.stdout:Discarding blocks...Done. 2024-03-13T21:32:10.144 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi023.front.sepia.ceph.com -o noatime 2024-03-13T21:32:10.144 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-13T21:32:10.193 DEBUG:teuthology.orchestra.run.smithi023:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-13T21:32:10.227 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-13T21:32:10.297 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-13T21:32:10.297 INFO:tasks.ceph:role: osd.2 2024-03-13T21:32:10.297 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi023.front.sepia.ceph.com 2024-03-13T21:32:10.297 DEBUG:teuthology.orchestra.run.smithi023:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-13T21:32:10.393 INFO:teuthology.orchestra.run.smithi023.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T21:32:10.393 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T21:32:10.394 INFO:teuthology.orchestra.run.smithi023.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T21:32:10.394 INFO:teuthology.orchestra.run.smithi023.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-13T21:32:10.394 INFO:teuthology.orchestra.run.smithi023.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T21:32:10.394 INFO:teuthology.orchestra.run.smithi023.stdout: = sunit=0 swidth=0 blks 2024-03-13T21:32:10.394 INFO:teuthology.orchestra.run.smithi023.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T21:32:10.394 INFO:teuthology.orchestra.run.smithi023.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-13T21:32:10.394 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T21:32:10.394 INFO:teuthology.orchestra.run.smithi023.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T21:32:11.679 INFO:teuthology.orchestra.run.smithi023.stdout:Discarding blocks...Done. 2024-03-13T21:32:11.681 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi023.front.sepia.ceph.com -o noatime 2024-03-13T21:32:11.681 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-13T21:32:11.729 DEBUG:teuthology.orchestra.run.smithi023:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-13T21:32:11.763 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-13T21:32:11.835 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-13T21:32:11.835 INFO:tasks.ceph:role: osd.3 2024-03-13T21:32:11.835 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi023.front.sepia.ceph.com 2024-03-13T21:32:11.835 DEBUG:teuthology.orchestra.run.smithi023:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-13T21:32:11.932 INFO:teuthology.orchestra.run.smithi023.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-13T21:32:11.932 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-13T21:32:11.932 INFO:teuthology.orchestra.run.smithi023.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-13T21:32:11.932 INFO:teuthology.orchestra.run.smithi023.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-13T21:32:11.932 INFO:teuthology.orchestra.run.smithi023.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-13T21:32:11.932 INFO:teuthology.orchestra.run.smithi023.stdout: = sunit=0 swidth=0 blks 2024-03-13T21:32:11.932 INFO:teuthology.orchestra.run.smithi023.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-13T21:32:11.932 INFO:teuthology.orchestra.run.smithi023.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-13T21:32:11.932 INFO:teuthology.orchestra.run.smithi023.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-13T21:32:11.932 INFO:teuthology.orchestra.run.smithi023.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-13T21:32:13.212 INFO:teuthology.orchestra.run.smithi023.stdout:Discarding blocks...Done. 2024-03-13T21:32:13.214 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi023.front.sepia.ceph.com -o noatime 2024-03-13T21:32:13.215 DEBUG:teuthology.orchestra.run.smithi023:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-13T21:32:13.262 DEBUG:teuthology.orchestra.run.smithi023:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-13T21:32:13.297 DEBUG:teuthology.orchestra.run.smithi023:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T21:32:13.394 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:13.402+0000 7f55e8dcef00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-13T21:32:13.394 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:13.403+0000 7f55e8dcef00 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-13T21:32:13.394 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:13.403+0000 7f55e8dcef00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-13T21:32:13.394 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:13.403+0000 7f55e8dcef00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-13T21:32:13.394 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:13.403+0000 7f55e8dcef00 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-13T21:32:13.395 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:13.403+0000 7f55e8dcef00 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-13T21:32:13.438 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:13.447+0000 7f55e8dcef00 -1 freelist read_size_meta_from_db missing size meta in DB 2024-03-13T21:32:15.760 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-13T21:32:15.792 DEBUG:teuthology.orchestra.run.smithi023:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T21:32:15.883 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:15.892+0000 7fe1823dff00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-13T21:32:15.883 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:15.892+0000 7fe1823dff00 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-13T21:32:15.883 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:15.892+0000 7fe1823dff00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-13T21:32:15.883 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:15.892+0000 7fe1823dff00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-13T21:32:15.883 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:15.892+0000 7fe1823dff00 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-13T21:32:15.883 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:15.893+0000 7fe1823dff00 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-13T21:32:15.903 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:15.913+0000 7fe1823dff00 -1 freelist read_size_meta_from_db missing size meta in DB 2024-03-13T21:32:18.240 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-13T21:32:18.272 DEBUG:teuthology.orchestra.run.smithi023:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T21:32:18.364 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:18.373+0000 7f93e4883f00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-13T21:32:18.364 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:18.373+0000 7f93e4883f00 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-13T21:32:18.364 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:18.373+0000 7f93e4883f00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-13T21:32:18.364 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:18.373+0000 7f93e4883f00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-13T21:32:18.364 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:18.373+0000 7f93e4883f00 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-13T21:32:18.365 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:18.374+0000 7f93e4883f00 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-13T21:32:18.385 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:18.394+0000 7f93e4883f00 -1 freelist read_size_meta_from_db missing size meta in DB 2024-03-13T21:32:20.706 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-13T21:32:20.740 DEBUG:teuthology.orchestra.run.smithi023:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-13T21:32:20.829 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:20.838+0000 7f2758cbbf00 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-13T21:32:20.829 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:20.838+0000 7f2758cbbf00 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-13T21:32:20.829 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:20.838+0000 7f2758cbbf00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-13T21:32:20.829 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:20.838+0000 7f2758cbbf00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-13T21:32:20.829 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:20.838+0000 7f2758cbbf00 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-13T21:32:20.829 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:20.839+0000 7f2758cbbf00 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-13T21:32:20.844 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:20.854+0000 7f2758cbbf00 -1 freelist read_size_meta_from_db missing size meta in DB 2024-03-13T21:32:23.173 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-13T21:32:23.205 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-13T21:32:23.206 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:23.206 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-13T21:32:23.277 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:23.277 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-13T21:32:23.348 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:23.348 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-13T21:32:23.418 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:23.418 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-13T21:32:23.488 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:23.488 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-13T21:32:23.556 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:23.556 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-13T21:32:23.625 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:23.625 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-13T21:32:23.694 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:23.694 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-13T21:32:23.763 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:23.763 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-13T21:32:23.830 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-03-13T21:32:23.830 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-13T21:32:23.854 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-03-13T21:32:23.854 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-13T21:32:23.877 INFO:tasks.ceph:Adding keys to all mons... 2024-03-13T21:32:23.877 DEBUG:teuthology.orchestra.run.smithi023:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-13T21:32:23.903 INFO:teuthology.orchestra.run.smithi023.stdout:[mgr.x] 2024-03-13T21:32:23.903 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBWG/JlPBkDHxAAQZnbdiCpYcn/XEz+CfG4FQ== 2024-03-13T21:32:23.903 INFO:teuthology.orchestra.run.smithi023.stdout:[mgr.y] 2024-03-13T21:32:23.903 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBWG/JlusMWIhAAN1Kwch4SfrI2HtVXIqwwhQ== 2024-03-13T21:32:23.903 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.a] 2024-03-13T21:32:23.903 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBWG/JlHtwuJRAAPVWwVK3+c8jaZ9QfCCF4fA== 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.b] 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBWG/Jl8mBILBAA2D9nrXkjPSTF89X7zbSvgw== 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout:[mds.c] 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBWG/Jl24JoMxAAeJUgko5Plz0Ey92MqD2Nug== 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.0] 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBdG/JlFfQUGBAA4wLGNP/h8cME41e4egvllw== 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.1] 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBfG/Jlyag9NRAAAw5wVGaMeZ3n66skZIET+Q== 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.2] 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBiG/JlGd5OFhAAiJ6vOCgTA6Y8peCLZ2JYkQ== 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout:[osd.3] 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBkG/JldYkFMhAAv+QIyt84TIMClbOh7TUQTQ== 2024-03-13T21:32:23.904 INFO:teuthology.orchestra.run.smithi023.stdout:[client.0] 2024-03-13T21:32:23.905 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBXG/JleAbCAxAAJpcrI+mXM31e28n40K/nFg== 2024-03-13T21:32:23.905 INFO:teuthology.orchestra.run.smithi023.stdout:[client.1] 2024-03-13T21:32:23.905 INFO:teuthology.orchestra.run.smithi023.stdout: key = AQBXG/JlrbKpCxAAwznz5c8OWXHbAE4iuKylRg== 2024-03-13T21:32:23.906 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-13T21:32:23.995 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-13T21:32:24.043 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-13T21:32:24.094 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-13T21:32:24.145 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-13T21:32:24.196 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T21:32:24.245 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T21:32:24.293 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T21:32:24.342 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-13T21:32:24.390 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-13T21:32:24.442 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-13T21:32:24.494 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-13T21:32:24.494 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-13T21:32:24.522 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-13T21:32:24.625 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-13T21:32:24.652 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-13T21:32:24.720 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-13T21:32:24.828 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-13T21:32:24.855 DEBUG:teuthology.orchestra.run.smithi023:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-13T21:32:24.923 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-13T21:32:25.023 DEBUG:teuthology.orchestra.run.smithi023:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-13T21:32:25.051 DEBUG:teuthology.orchestra.run.smithi023:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-13T21:32:25.108 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-13T21:32:25.109 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-13T21:32:25.109 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-13T21:32:25.151 INFO:tasks.ceph.mon.a:Started 2024-03-13T21:32:25.152 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-13T21:32:25.152 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-13T21:32:25.154 INFO:tasks.ceph.mon.b:Started 2024-03-13T21:32:25.155 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-13T21:32:25.155 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-13T21:32:25.157 INFO:tasks.ceph.mon.c:Started 2024-03-13T21:32:25.158 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-13T21:32:25.158 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-13T21:32:25.158 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-13T21:32:25.161 INFO:tasks.ceph.mgr.x:Started 2024-03-13T21:32:25.162 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-13T21:32:25.162 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-13T21:32:25.164 INFO:tasks.ceph.mgr.y:Started 2024-03-13T21:32:25.165 INFO:tasks.ceph:Setting crush tunables to default 2024-03-13T21:32:25.165 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --cluster ceph osd crush tunables default 2024-03-13T21:32:25.296 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51c5b73700 1 Processor -- start 2024-03-13T21:32:25.296 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51c5b73700 1 -- start start 2024-03-13T21:32:25.296 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51c5b73700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:25.296 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51c5b73700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51c5b73700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51c5b73700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c005b970 con 0x7f51c0136bf0 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51c5b73700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0136710 con 0x7f51c005d370 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51c5b73700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51c01369d0 con 0x7f51c0136210 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51bffff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51bf7fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51bffff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:54716/0 (socket says 172.21.15.23:54716) 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51bffff700 1 -- 172.21.15.23:0/1929557940 learned_addr learned my addr 172.21.15.23:0/1929557940 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.304+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51bffff700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 msgr2=0x7f51c01397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-13T21:32:25.297 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51bffff700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 msgr2=0x7f51c01397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:25.298 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:25.298 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51bf7fe700 1 -- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 msgr2=0x7f51c005bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-13T21:32:25.298 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51bf7fe700 1 -- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 msgr2=0x7f51c005bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:25.298 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 msgr2=0x7f51c005ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-13T21:32:25.298 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 msgr2=0x7f51c005ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:25.298 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:25.298 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:25.298 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-03-13T21:32:25.298 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-03-13T21:32:25.298 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.305+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-03-13T21:32:25.498 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:25.498 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:25.498 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:25.499 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51bf7fe700 1 -- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 msgr2=0x7f51c005bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-13T21:32:25.499 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51bf7fe700 1 -- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 msgr2=0x7f51c005bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:25.499 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:25.499 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51bffff700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 msgr2=0x7f51c01397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-13T21:32:25.499 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51bffff700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 msgr2=0x7f51c01397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:25.499 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 msgr2=0x7f51c005ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-13T21:32:25.500 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 msgr2=0x7f51c005ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:25.500 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2024-03-13T21:32:25.500 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:25.500 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:25.501 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2024-03-13T21:32:25.501 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.506+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2024-03-13T21:32:25.898 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:25.898 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 msgr2=0x7f51c005ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51bf7fe700 1 -- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 msgr2=0x7f51c005bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 msgr2=0x7f51c005ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51bf7fe700 1 -- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 msgr2=0x7f51c005bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51bffff700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 msgr2=0x7f51c01397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51bffff700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 msgr2=0x7f51c01397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:25.899 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:25.907+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2024-03-13T21:32:26.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:26.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:26.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:26.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51bf7fe700 1 -- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 msgr2=0x7f51c005bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-13T21:32:26.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51bf7fe700 1 -- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 msgr2=0x7f51c005bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:26.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:26.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51bffff700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 msgr2=0x7f51c01397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-13T21:32:26.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51bffff700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 msgr2=0x7f51c01397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:26.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:26.701 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2024-03-13T21:32:26.701 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2024-03-13T21:32:26.701 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 msgr2=0x7f51c005ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-13T21:32:26.701 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 msgr2=0x7f51c005ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-13T21:32:26.701 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-13T21:32:26.701 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:26.708+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2024-03-13T21:32:28.295 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:28.304+0000 7f51bd7fa700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 msgr2=0x7f51c005ce80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T21:32:28.296 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:28.304+0000 7f51bd7fa700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51c005ce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:28.296 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:28.304+0000 7f51bd7fa700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 msgr2=0x7f51c01397e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T21:32:28.296 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:28.304+0000 7f51bd7fa700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51c01397e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:28.296 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:28.304+0000 7f51bd7fa700 1 -- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 msgr2=0x7f51c005bde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-13T21:32:28.296 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:28.304+0000 7f51bd7fa700 1 --2- 172.21.15.23:0/1929557940 >> v2:172.21.15.23:3300/0 conn(0x7f51c0136bf0 0x7f51c005bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:28.296 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:28.304+0000 7f51bd7fa700 1 -- 172.21.15.23:0/1929557940 --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c0136fd0 con 0x7f51a00033b0 2024-03-13T21:32:30.550 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.558+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241276519 0 0) 0x7f51c0136fd0 con 0x7f51a00033b0 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.558+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 --> v1:172.21.15.23:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a4003260 con 0x7f51a00033b0 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.558+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 <== mon.0 v1:172.21.15.23:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3043591045 0 0) 0x7f51a4003260 con 0x7f51a00033b0 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.558+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 --> v1:172.21.15.23:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0008050 con 0x7f51a00033b0 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.558+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 <== mon.0 v1:172.21.15.23:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 3360728538 0 0) 0x7f51b000d1c0 con 0x7f51a00033b0 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.558+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 >> v1:172.21.15.23:6789/0 conn(0x7f51a00033b0 legacy=0x7f51a0005870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.558+0000 7f51be7fc700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51a40037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51be7fc700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 0x7f51a4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51be7fc700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51a4004b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4005210 con 0x7f51a4003e70 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4005380 con 0x7f51c005d370 2024-03-13T21:32:30.551 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51a400b780 con 0x7f51c0136210 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51bffff700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51a4004b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 0x7f51a4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51a40037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 msgr2=0x7f51a40037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51a40037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 msgr2=0x7f51a4004b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c005d370 0x7f51a4004b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51beffd700 1 -- 172.21.15.23:0/1929557940 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4003260 con 0x7f51a4003e70 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c0136210 0x7f51a40037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.559+0000 7f51beffd700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 0x7f51a4004500 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f51a800a000 tx=0x7f51a8008040).ready entity=mon.0 client_cookie=58cfe6df0f3e3b28 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.560+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a8014a50 con 0x7f51a4003e70 2024-03-13T21:32:30.552 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.560+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a800c040 con 0x7f51a4003e70 2024-03-13T21:32:30.553 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.560+0000 7f51be7fc700 1 -- 172.21.15.23:0/1929557940 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a8014da0 con 0x7f51a4003e70 2024-03-13T21:32:30.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.560+0000 7f51c5b73700 1 -- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 msgr2=0x7f51a4004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:30.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.560+0000 7f51c5b73700 1 --2- 172.21.15.23:0/1929557940 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 0x7f51a4004500 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f51a800a000 tx=0x7f51a8008040).stop 2024-03-13T21:32:30.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.560+0000 7f51bffff700 1 -- 172.21.15.23:0/1929557940 reap_dead start 2024-03-13T21:32:30.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.560+0000 7f51c5b73700 1 -- 172.21.15.23:0/1929557940 shutdown_connections 2024-03-13T21:32:30.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.560+0000 7f51c5b73700 1 -- 172.21.15.23:0/1929557940 >> 172.21.15.23:0/1929557940 conn(0x7f51c0066a50 msgr2=0x7f51c006c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:30.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.560+0000 7f51c5b73700 1 -- 172.21.15.23:0/1929557940 shutdown_connections 2024-03-13T21:32:30.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.560+0000 7f51c5b73700 1 -- 172.21.15.23:0/1929557940 wait complete. 2024-03-13T21:32:30.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.561+0000 7f51c5b73700 1 Processor -- start 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.562+0000 7f51c5b73700 1 -- start start 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.562+0000 7f51c5b73700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 0x7f51c013d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.562+0000 7f51c5b73700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c00a1d90 0x7f51c00ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.562+0000 7f51c5b73700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c012f690 0x7f51c012fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.562+0000 7f51c5b73700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0137450 con 0x7f51a4003e70 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.562+0000 7f51c5b73700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00a2500 con 0x7f51c012f690 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.562+0000 7f51c5b73700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00a2240 con 0x7f51c00a1d90 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51bf7fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 0x7f51c013d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51beffd700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c00a1d90 0x7f51c00ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51bf7fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 0x7f51c013d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:44098/0 (socket says 172.21.15.23:44098) 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51bffff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c012f690 0x7f51c012fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51bf7fe700 1 -- 172.21.15.23:0/2988505625 learned_addr learned my addr 172.21.15.23:0/2988505625 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51beffd700 1 -- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c012f690 msgr2=0x7f51c012fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:30.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51beffd700 1 --2- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c012f690 0x7f51c012fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51beffd700 1 -- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 msgr2=0x7f51c013d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51beffd700 1 --2- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 0x7f51c013d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51beffd700 1 -- 172.21.15.23:0/2988505625 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00ac510 con 0x7f51c00a1d90 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51bf7fe700 1 --2- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 0x7f51c013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51beffd700 1 --2- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c00a1d90 0x7f51c00ac020 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f51a800a000 tx=0x7f51a8008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51bcff9700 1 -- 172.21.15.23:0/2988505625 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a8026070 con 0x7f51c00a1d90 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51bcff9700 1 -- 172.21.15.23:0/2988505625 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a8017870 con 0x7f51c00a1d90 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.563+0000 7f51c5b73700 1 -- 172.21.15.23:0/2988505625 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c0130000 con 0x7f51c00a1d90 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.564+0000 7f51bcff9700 1 -- 172.21.15.23:0/2988505625 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a8017bd0 con 0x7f51c00a1d90 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.564+0000 7f51c5b73700 1 -- 172.21.15.23:0/2988505625 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c00a0e20 con 0x7f51c00a1d90 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.564+0000 7f51bcff9700 1 -- 172.21.15.23:0/2988505625 <== mon.2 v2:172.21.15.23:3302/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f51a801d030 con 0x7f51c00a1d90 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.564+0000 7f51bcff9700 1 -- 172.21.15.23:0/2988505625 <== mon.2 v2:172.21.15.23:3302/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f51a802b9b0 con 0x7f51c00a1d90 2024-03-13T21:32:30.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.565+0000 7f51c5b73700 1 -- 172.21.15.23:0/2988505625 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c00a2240 con 0x7f51c00a1d90 2024-03-13T21:32:30.559 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.567+0000 7f51bcff9700 1 -- 172.21.15.23:0/2988505625 <== mon.2 v2:172.21.15.23:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f51a8030020 con 0x7f51c00a1d90 2024-03-13T21:32:30.629 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.638+0000 7f51c5b73700 1 -- 172.21.15.23:0/2988505625 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f51c00a1040 con 0x7f51c00a1d90 2024-03-13T21:32:30.633 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.642+0000 7f51bcff9700 1 -- 172.21.15.23:0/2988505625 <== mon.2 v2:172.21.15.23:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f51a801b070 con 0x7f51c00a1d90 2024-03-13T21:32:30.634 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.643+0000 7f51c5b73700 1 -- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c00a1d90 msgr2=0x7f51c00ac020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:30.634 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.643+0000 7f51c5b73700 1 --2- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c00a1d90 0x7f51c00ac020 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f51a800a000 tx=0x7f51a8008040).stop 2024-03-13T21:32:30.634 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.643+0000 7f51c5b73700 1 -- 172.21.15.23:0/2988505625 shutdown_connections 2024-03-13T21:32:30.634 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.643+0000 7f51c5b73700 1 --2- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f51c012f690 0x7f51c012fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:30.634 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.643+0000 7f51c5b73700 1 --2- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f51a4003e70 0x7f51c013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:30.635 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.643+0000 7f51c5b73700 1 --2- 172.21.15.23:0/2988505625 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f51c00a1d90 0x7f51c00ac020 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:30.635 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.643+0000 7f51c5b73700 1 -- 172.21.15.23:0/2988505625 >> 172.21.15.23:0/2988505625 conn(0x7f51c0066a50 msgr2=0x7f51c00a9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:30.635 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.643+0000 7f51c5b73700 1 -- 172.21.15.23:0/2988505625 shutdown_connections 2024-03-13T21:32:30.635 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:30.643+0000 7f51c5b73700 1 -- 172.21.15.23:0/2988505625 wait complete. 2024-03-13T21:32:30.635 INFO:teuthology.orchestra.run.smithi023.stderr:adjusted tunables profile to default 2024-03-13T21:32:30.645 INFO:tasks.ceph:check_enable_crimson: False 2024-03-13T21:32:30.645 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-13T21:32:30.645 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:30.645 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-13T21:32:30.673 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:30.673 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-13T21:32:30.741 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:30.741 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-13T21:32:30.809 DEBUG:teuthology.orchestra.run.smithi023:> set -ex 2024-03-13T21:32:30.809 DEBUG:teuthology.orchestra.run.smithi023:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-13T21:32:30.876 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --cluster ceph osd new a63e0271-0bb4-4f3b-b8a0-681e5b8ce56b 0 2024-03-13T21:32:31.020 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.029+0000 7f0f535d2700 1 Processor -- start 2024-03-13T21:32:31.020 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f535d2700 1 -- start start 2024-03-13T21:32:31.023 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f535d2700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 0x7f0f4c09b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.023 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f535d2700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f0f4c068080 0x7f0f4c140900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.023 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f535d2700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4c066c90 con 0x7f0f4c132fc0 2024-03-13T21:32:31.023 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f535d2700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c133ab0 con 0x7f0f4c068080 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f535d2700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c09b5e0 con 0x7f0f4c0679a0 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f51b6f700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f0f4c068080 0x7f0f4c140900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f50b6d700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 0x7f0f4c09b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f51b6f700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f0f4c068080 0x7f0f4c140900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:44102/0 (socket says 172.21.15.23:44102) 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f51b6f700 1 -- 172.21.15.23:0/4215217301 learned_addr learned my addr 172.21.15.23:0/4215217301 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.030+0000 7f0f5136e700 1 --1- 172.21.15.23:0/4215217301 >> v1:172.21.15.23:6789/0 conn(0x7f0f4c132fc0 0x7f0f4c1333a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:49830/0 (socket says 172.21.15.23:49830) 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f51b6f700 1 -- 172.21.15.23:0/4215217301 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 msgr2=0x7f0f4c09b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f51b6f700 1 --2- 172.21.15.23:0/4215217301 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 0x7f0f4c09b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f51b6f700 1 -- 172.21.15.23:0/4215217301 >> v1:172.21.15.23:6789/0 conn(0x7f0f4c132fc0 legacy=0x7f0f4c1333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f51b6f700 1 -- 172.21.15.23:0/4215217301 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c09b880 con 0x7f0f4c068080 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f3ffff700 1 -- 172.21.15.23:0/4215217301 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806820310 0 0) 0x7f0f4c066c90 con 0x7f0f4c132fc0 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f50b6d700 1 --2- 172.21.15.23:0/4215217301 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 0x7f0f4c09b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f51b6f700 1 --2- 172.21.15.23:0/4215217301 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 0x7f0f4c140900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f0f440032f0 tx=0x7f0f44009290).ready entity=mon.0 client_cookie=cba114d5ca3ca793 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:31.024 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f3ffff700 1 -- 172.21.15.23:0/4215217301 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f4400b8a0 con 0x7f0f4c068080 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f3ffff700 1 -- 172.21.15.23:0/4215217301 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4400ba00 con 0x7f0f4c068080 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f3ffff700 1 -- 172.21.15.23:0/4215217301 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f4400bb70 con 0x7f0f4c068080 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f535d2700 1 -- 172.21.15.23:0/4215217301 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 msgr2=0x7f0f4c140900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f535d2700 1 --2- 172.21.15.23:0/4215217301 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 0x7f0f4c140900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f0f440032f0 tx=0x7f0f44009290).stop 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f535d2700 1 -- 172.21.15.23:0/4215217301 shutdown_connections 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f535d2700 1 --2- 172.21.15.23:0/4215217301 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 0x7f0f4c140900 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f535d2700 1 --2- 172.21.15.23:0/4215217301 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 0x7f0f4c09b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.031+0000 7f0f535d2700 1 -- 172.21.15.23:0/4215217301 >> 172.21.15.23:0/4215217301 conn(0x7f0f4c05cf10 msgr2=0x7f0f4c05d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.032+0000 7f0f535d2700 1 -- 172.21.15.23:0/4215217301 shutdown_connections 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.032+0000 7f0f535d2700 1 -- 172.21.15.23:0/4215217301 wait complete. 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.032+0000 7f0f535d2700 1 Processor -- start 2024-03-13T21:32:31.025 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.032+0000 7f0f535d2700 1 -- start start 2024-03-13T21:32:31.028 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f535d2700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 0x7f0f4c135bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.028 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f535d2700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 0x7f0f4c1353e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.028 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f535d2700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0f4c132fc0 0x7f0f4c136970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.028 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f535d2700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c13c5f0 con 0x7f0f4c068080 2024-03-13T21:32:31.028 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f535d2700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c092f90 con 0x7f0f4c0679a0 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f535d2700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c093250 con 0x7f0f4c132fc0 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f50b6d700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 0x7f0f4c1353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f50b6d700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 0x7f0f4c1353e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:44118/0 (socket says 172.21.15.23:44118) 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f50b6d700 1 -- 172.21.15.23:0/3387370675 learned_addr learned my addr 172.21.15.23:0/3387370675 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f51b6f700 1 --2- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0f4c132fc0 0x7f0f4c136970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f50b6d700 1 -- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0f4c132fc0 msgr2=0x7f0f4c136970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f5136e700 1 --2- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 0x7f0f4c135bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f50b6d700 1 --2- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0f4c132fc0 0x7f0f4c136970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f50b6d700 1 -- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 msgr2=0x7f0f4c135bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f50b6d700 1 --2- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 0x7f0f4c135bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f50b6d700 1 -- 172.21.15.23:0/3387370675 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4000b180 con 0x7f0f4c068080 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f5136e700 1 --2- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 0x7f0f4c135bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.033+0000 7f0f50b6d700 1 --2- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 0x7f0f4c1353e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f0f4000c000 tx=0x7f0f40006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:31.029 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.034+0000 7f0f3e7fc700 1 -- 172.21.15.23:0/3387370675 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f4000c890 con 0x7f0f4c068080 2024-03-13T21:32:31.030 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.034+0000 7f0f3e7fc700 1 -- 172.21.15.23:0/3387370675 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4000c9f0 con 0x7f0f4c068080 2024-03-13T21:32:31.030 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.034+0000 7f0f535d2700 1 -- 172.21.15.23:0/3387370675 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f44009530 con 0x7f0f4c068080 2024-03-13T21:32:31.030 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.034+0000 7f0f3e7fc700 1 -- 172.21.15.23:0/3387370675 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f4000cd50 con 0x7f0f4c068080 2024-03-13T21:32:31.030 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.034+0000 7f0f535d2700 1 -- 172.21.15.23:0/3387370675 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4c13bcd0 con 0x7f0f4c068080 2024-03-13T21:32:31.030 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.034+0000 7f0f3e7fc700 1 -- 172.21.15.23:0/3387370675 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f0f4001c630 con 0x7f0f4c068080 2024-03-13T21:32:31.030 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.034+0000 7f0f3e7fc700 1 -- 172.21.15.23:0/3387370675 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0f40012740 con 0x7f0f4c068080 2024-03-13T21:32:31.030 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.035+0000 7f0f535d2700 1 -- 172.21.15.23:0/3387370675 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4c09a630 con 0x7f0f4c068080 2024-03-13T21:32:31.030 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.037+0000 7f0f3e7fc700 1 -- 172.21.15.23:0/3387370675 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f0f4001cd70 con 0x7f0f4c068080 2024-03-13T21:32:31.103 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.113+0000 7f0f535d2700 1 -- 172.21.15.23:0/3387370675 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a63e0271-0bb4-4f3b-b8a0-681e5b8ce56b", "id": 0} v 0) v1 -- 0x7f0f4c057a60 con 0x7f0f4c068080 2024-03-13T21:32:31.109 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.118+0000 7f0f3e7fc700 1 -- 172.21.15.23:0/3387370675 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a63e0271-0bb4-4f3b-b8a0-681e5b8ce56b", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0f40012900 con 0x7f0f4c068080 2024-03-13T21:32:31.109 INFO:teuthology.orchestra.run.smithi023.stdout:0 2024-03-13T21:32:31.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.119+0000 7f0f535d2700 1 -- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 msgr2=0x7f0f4c1353e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.119+0000 7f0f535d2700 1 --2- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 0x7f0f4c1353e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f0f4000c000 tx=0x7f0f40006df0).stop 2024-03-13T21:32:31.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.119+0000 7f0f535d2700 1 -- 172.21.15.23:0/3387370675 shutdown_connections 2024-03-13T21:32:31.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.119+0000 7f0f535d2700 1 --2- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0f4c132fc0 0x7f0f4c136970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.119+0000 7f0f535d2700 1 --2- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0f4c0679a0 0x7f0f4c135bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.119+0000 7f0f535d2700 1 --2- 172.21.15.23:0/3387370675 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0f4c068080 0x7f0f4c1353e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.119+0000 7f0f535d2700 1 -- 172.21.15.23:0/3387370675 >> 172.21.15.23:0/3387370675 conn(0x7f0f4c05cf10 msgr2=0x7f0f4c099930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:31.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.119+0000 7f0f535d2700 1 -- 172.21.15.23:0/3387370675 shutdown_connections 2024-03-13T21:32:31.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.119+0000 7f0f535d2700 1 -- 172.21.15.23:0/3387370675 wait complete. 2024-03-13T21:32:31.120 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --cluster ceph osd new a721ee5e-d9d6-420e-ba1f-78ff60306e44 1 2024-03-13T21:32:31.225 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.234+0000 7f2e73f91700 1 Processor -- start 2024-03-13T21:32:31.225 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.234+0000 7f2e73f91700 1 -- start start 2024-03-13T21:32:31.228 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.234+0000 7f2e73f91700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c0a33f0 0x7f2e6c0a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.228 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.234+0000 7f2e73f91700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c13b1f0 0x7f2e6c13d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.228 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.234+0000 7f2e73f91700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6c0a01e0 con 0x7f2e6c0a37d0 2024-03-13T21:32:31.228 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.234+0000 7f2e73f91700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0a0480 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.228 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.234+0000 7f2e73f91700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0a2000 con 0x7f2e6c0a33f0 2024-03-13T21:32:31.228 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e7252e700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c13b1f0 0x7f2e6c13d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.228 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e71d2d700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c0a33f0 0x7f2e6c0a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.228 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e7252e700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c13b1f0 0x7f2e6c13d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:41058/0 (socket says 172.21.15.23:41058) 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e71d2d700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c0a33f0 0x7f2e6c0a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:52474/0 (socket says 172.21.15.23:52474) 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e7252e700 1 -- 172.21.15.23:0/3078924935 learned_addr learned my addr 172.21.15.23:0/3078924935 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e70d2b700 1 -- 172.21.15.23:0/3078924935 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492135269 0 0) 0x7f2e6c0a01e0 con 0x7f2e6c0a37d0 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e71d2d700 1 -- 172.21.15.23:0/3078924935 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c13b1f0 msgr2=0x7f2e6c13d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e71d2d700 1 --2- 172.21.15.23:0/3078924935 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c13b1f0 0x7f2e6c13d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e71d2d700 1 -- 172.21.15.23:0/3078924935 >> v1:172.21.15.23:6789/0 conn(0x7f2e6c0a37d0 legacy=0x7f2e6c0a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e71d2d700 1 -- 172.21.15.23:0/3078924935 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c09ff10 con 0x7f2e6c0a33f0 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e7252e700 1 --2- 172.21.15.23:0/3078924935 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c13b1f0 0x7f2e6c13d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e71d2d700 1 --2- 172.21.15.23:0/3078924935 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c0a33f0 0x7f2e6c0a1b10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f2e68002f70 tx=0x7f2e68006e80).ready entity=mon.2 client_cookie=e84fba94f389ee80 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e70d2b700 1 -- 172.21.15.23:0/3078924935 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e68014070 con 0x7f2e6c0a33f0 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e70d2b700 1 -- 172.21.15.23:0/3078924935 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e68000e90 con 0x7f2e6c0a33f0 2024-03-13T21:32:31.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.235+0000 7f2e70d2b700 1 -- 172.21.15.23:0/3078924935 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e680107f0 con 0x7f2e6c0a33f0 2024-03-13T21:32:31.230 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.236+0000 7f2e73f91700 1 -- 172.21.15.23:0/3078924935 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c0a33f0 msgr2=0x7f2e6c0a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.230 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.236+0000 7f2e73f91700 1 --2- 172.21.15.23:0/3078924935 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c0a33f0 0x7f2e6c0a1b10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f2e68002f70 tx=0x7f2e68006e80).stop 2024-03-13T21:32:31.230 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.236+0000 7f2e73f91700 1 -- 172.21.15.23:0/3078924935 shutdown_connections 2024-03-13T21:32:31.230 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.236+0000 7f2e73f91700 1 --2- 172.21.15.23:0/3078924935 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c13b1f0 0x7f2e6c13d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.230 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.236+0000 7f2e73f91700 1 --2- 172.21.15.23:0/3078924935 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c0a33f0 0x7f2e6c0a1b10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.230 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.236+0000 7f2e73f91700 1 -- 172.21.15.23:0/3078924935 >> 172.21.15.23:0/3078924935 conn(0x7f2e6c0a89d0 msgr2=0x7f2e6c0a8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:31.230 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.236+0000 7f2e73f91700 1 -- 172.21.15.23:0/3078924935 shutdown_connections 2024-03-13T21:32:31.230 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.236+0000 7f2e73f91700 1 -- 172.21.15.23:0/3078924935 wait complete. 2024-03-13T21:32:31.230 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.237+0000 7f2e73f91700 1 Processor -- start 2024-03-13T21:32:31.230 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.237+0000 7f2e73f91700 1 -- start start 2024-03-13T21:32:31.232 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.237+0000 7f2e73f91700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2e6c0a33f0 0x7f2e6c09e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.232 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.237+0000 7f2e73f91700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c0a37d0 0x7f2e6c1341a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.237+0000 7f2e73f91700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c13b1f0 0x7f2e6c134690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e73f91700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0a52e0 con 0x7f2e6c0a33f0 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e73f91700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c065400 con 0x7f2e6c0a37d0 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e73f91700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c064d00 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e71d2d700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2e6c0a33f0 0x7f2e6c09e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e7252e700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c13b1f0 0x7f2e6c134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e71d2d700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2e6c0a33f0 0x7f2e6c09e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:44128/0 (socket says 172.21.15.23:44128) 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e71d2d700 1 -- 172.21.15.23:0/2653773284 learned_addr learned my addr 172.21.15.23:0/2653773284 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e7152c700 1 --2- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c0a37d0 0x7f2e6c1341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e7252e700 1 -- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c0a37d0 msgr2=0x7f2e6c1341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e7252e700 1 --2- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c0a37d0 0x7f2e6c1341a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e7252e700 1 -- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2e6c0a33f0 msgr2=0x7f2e6c09e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e7252e700 1 --2- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2e6c0a33f0 0x7f2e6c09e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e7252e700 1 -- 172.21.15.23:0/2653773284 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64009530 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.233 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e71d2d700 1 --2- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2e6c0a33f0 0x7f2e6c09e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:31.234 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e7252e700 1 --2- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c13b1f0 0x7f2e6c134690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f2e6400c000 tx=0x7f2e64009290).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:31.234 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e5affd700 1 -- 172.21.15.23:0/2653773284 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e6400c820 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.234 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e5affd700 1 -- 172.21.15.23:0/2653773284 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6400c980 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.234 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e73f91700 1 -- 172.21.15.23:0/2653773284 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6800c310 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.234 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e5affd700 1 -- 172.21.15.23:0/2653773284 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e6400cce0 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.234 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.238+0000 7f2e73f91700 1 -- 172.21.15.23:0/2653773284 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6c09ed80 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.234 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.239+0000 7f2e5affd700 1 -- 172.21.15.23:0/2653773284 <== mon.2 v2:172.21.15.23:3302/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f2e64022070 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.234 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.239+0000 7f2e5affd700 1 -- 172.21.15.23:0/2653773284 <== mon.2 v2:172.21.15.23:3302/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f2e6400ce40 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.234 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.239+0000 7f2e73f91700 1 -- 172.21.15.23:0/2653773284 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6c064ec0 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.234 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.242+0000 7f2e5affd700 1 -- 172.21.15.23:0/2653773284 <== mon.2 v2:172.21.15.23:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f2e64039870 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.311 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.320+0000 7f2e73f91700 1 -- 172.21.15.23:0/2653773284 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "osd new", "uuid": "a721ee5e-d9d6-420e-ba1f-78ff60306e44", "id": 1} v 0) v1 -- 0x7f2e6c0a4760 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.319 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.328+0000 7f2e5affd700 1 -- 172.21.15.23:0/2653773284 <== mon.2 v2:172.21.15.23:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a721ee5e-d9d6-420e-ba1f-78ff60306e44", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2e64029030 con 0x7f2e6c13b1f0 2024-03-13T21:32:31.320 INFO:teuthology.orchestra.run.smithi023.stdout:1 2024-03-13T21:32:31.321 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.330+0000 7f2e73f91700 1 -- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c13b1f0 msgr2=0x7f2e6c134690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.321 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.330+0000 7f2e73f91700 1 --2- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c13b1f0 0x7f2e6c134690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f2e6400c000 tx=0x7f2e64009290).stop 2024-03-13T21:32:31.321 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.330+0000 7f2e73f91700 1 -- 172.21.15.23:0/2653773284 shutdown_connections 2024-03-13T21:32:31.321 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.330+0000 7f2e73f91700 1 --2- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2e6c13b1f0 0x7f2e6c134690 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.321 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.330+0000 7f2e73f91700 1 --2- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2e6c0a33f0 0x7f2e6c09e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.321 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.330+0000 7f2e73f91700 1 --2- 172.21.15.23:0/2653773284 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2e6c0a37d0 0x7f2e6c1341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.322 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.330+0000 7f2e73f91700 1 -- 172.21.15.23:0/2653773284 >> 172.21.15.23:0/2653773284 conn(0x7f2e6c0a89d0 msgr2=0x7f2e6c05dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:31.322 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.330+0000 7f2e73f91700 1 -- 172.21.15.23:0/2653773284 shutdown_connections 2024-03-13T21:32:31.322 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.330+0000 7f2e73f91700 1 -- 172.21.15.23:0/2653773284 wait complete. 2024-03-13T21:32:31.331 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --cluster ceph osd new 9ce90734-afdb-498c-952c-64a477a8f27a 2 2024-03-13T21:32:31.439 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.448+0000 7f55e2138700 1 Processor -- start 2024-03-13T21:32:31.439 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.448+0000 7f55e2138700 1 -- start start 2024-03-13T21:32:31.442 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55e2138700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f55dc139bb0 0x7f55dc065ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.442 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55e2138700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc066c90 0x7f55dc0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.442 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55e2138700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139f90 0x7f55dc05b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.442 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55e2138700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc0a4da0 con 0x7f55dc139bb0 2024-03-13T21:32:31.442 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55e2138700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc065fb0 con 0x7f55dc139f90 2024-03-13T21:32:31.442 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55e2138700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc066520 con 0x7f55dc066c90 2024-03-13T21:32:31.442 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55db7fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f55dc139bb0 0x7f55dc065ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.442 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55db7fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f55dc139bb0 0x7f55dc065ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:44176/0 (socket says 172.21.15.23:44176) 2024-03-13T21:32:31.442 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55daffd700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc066c90 0x7f55dc0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55db7fe700 1 -- 172.21.15.23:0/4115675499 learned_addr learned my addr 172.21.15.23:0/4115675499 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55dbfff700 1 --2- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139f90 0x7f55dc05b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55db7fe700 1 -- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc066c90 msgr2=0x7f55dc0a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55db7fe700 1 --2- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc066c90 0x7f55dc0a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55db7fe700 1 -- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139f90 msgr2=0x7f55dc05b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55db7fe700 1 --2- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139f90 0x7f55dc05b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55db7fe700 1 -- 172.21.15.23:0/4115675499 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc0a4b00 con 0x7f55dc139bb0 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.449+0000 7f55daffd700 1 --2- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc066c90 0x7f55dc0a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.450+0000 7f55db7fe700 1 --2- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f55dc139bb0 0x7f55dc065ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f55dc0666b0 tx=0x7f55cc006eb0).ready entity=mon.0 client_cookie=74eac3126db09b05 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.450+0000 7f55da7fc700 1 -- 172.21.15.23:0/4115675499 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55cc00d570 con 0x7f55dc139bb0 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.450+0000 7f55da7fc700 1 -- 172.21.15.23:0/4115675499 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc000e90 con 0x7f55dc139bb0 2024-03-13T21:32:31.443 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.450+0000 7f55da7fc700 1 -- 172.21.15.23:0/4115675499 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55cc00d8c0 con 0x7f55dc139bb0 2024-03-13T21:32:31.444 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.450+0000 7f55e2138700 1 -- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f55dc139bb0 msgr2=0x7f55dc065ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.444 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.450+0000 7f55e2138700 1 --2- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f55dc139bb0 0x7f55dc065ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f55dc0666b0 tx=0x7f55cc006eb0).stop 2024-03-13T21:32:31.444 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.451+0000 7f55e2138700 1 -- 172.21.15.23:0/4115675499 shutdown_connections 2024-03-13T21:32:31.444 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.451+0000 7f55e2138700 1 --2- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139f90 0x7f55dc05b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.444 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.451+0000 7f55e2138700 1 --2- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f55dc139bb0 0x7f55dc065ac0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.444 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.451+0000 7f55e2138700 1 --2- 172.21.15.23:0/4115675499 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc066c90 0x7f55dc0a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.444 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.451+0000 7f55e2138700 1 -- 172.21.15.23:0/4115675499 >> 172.21.15.23:0/4115675499 conn(0x7f55dc0a89d0 msgr2=0x7f55dc0a8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:31.444 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.451+0000 7f55e2138700 1 -- 172.21.15.23:0/4115675499 shutdown_connections 2024-03-13T21:32:31.444 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.451+0000 7f55e2138700 1 -- 172.21.15.23:0/4115675499 wait complete. 2024-03-13T21:32:31.446 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.452+0000 7f55e2138700 1 Processor -- start 2024-03-13T21:32:31.446 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.452+0000 7f55e2138700 1 -- start start 2024-03-13T21:32:31.446 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55e2138700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f55dc066c90 0x7f55dc092280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55e2138700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139bb0 0x7f55dc148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55e2138700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc139f90 0x7f55dc149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55e2138700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc093ad0 con 0x7f55dc066c90 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55e2138700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc092b40 con 0x7f55dc139bb0 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55e2138700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0a0810 con 0x7f55dc139f90 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55daffd700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139bb0 0x7f55dc148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55daffd700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139bb0 0x7f55dc148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:41110/0 (socket says 172.21.15.23:41110) 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55daffd700 1 -- 172.21.15.23:0/821227821 learned_addr learned my addr 172.21.15.23:0/821227821 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55dbfff700 1 --2- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc139f90 0x7f55dc149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55db7fe700 1 --2- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f55dc066c90 0x7f55dc092280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55db7fe700 1 -- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc139f90 msgr2=0x7f55dc149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55db7fe700 1 --2- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc139f90 0x7f55dc149270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55db7fe700 1 -- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139bb0 msgr2=0x7f55dc148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55db7fe700 1 --2- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139bb0 0x7f55dc148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.447 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55db7fe700 1 -- 172.21.15.23:0/821227821 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d000b180 con 0x7f55dc066c90 2024-03-13T21:32:31.448 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55dbfff700 1 --2- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc139f90 0x7f55dc149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:31.448 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55db7fe700 1 --2- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f55dc066c90 0x7f55dc092280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f55dc0666b0 tx=0x7f55cc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:31.448 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.453+0000 7f55d8ff9700 1 -- 172.21.15.23:0/821227821 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55cc002f70 con 0x7f55dc066c90 2024-03-13T21:32:31.448 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.454+0000 7f55d8ff9700 1 -- 172.21.15.23:0/821227821 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc011970 con 0x7f55dc066c90 2024-03-13T21:32:31.448 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.454+0000 7f55e2138700 1 -- 172.21.15.23:0/821227821 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc00b1e0 con 0x7f55dc066c90 2024-03-13T21:32:31.448 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.454+0000 7f55d8ff9700 1 -- 172.21.15.23:0/821227821 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55cc011cd0 con 0x7f55dc066c90 2024-03-13T21:32:31.448 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.454+0000 7f55e2138700 1 -- 172.21.15.23:0/821227821 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc0a21b0 con 0x7f55dc066c90 2024-03-13T21:32:31.448 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.454+0000 7f55d8ff9700 1 -- 172.21.15.23:0/821227821 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f55cc024030 con 0x7f55dc066c90 2024-03-13T21:32:31.448 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.454+0000 7f55d8ff9700 1 -- 172.21.15.23:0/821227821 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f55cc0267e0 con 0x7f55dc066c90 2024-03-13T21:32:31.448 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.455+0000 7f55e2138700 1 -- 172.21.15.23:0/821227821 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55dc094bf0 con 0x7f55dc066c90 2024-03-13T21:32:31.449 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.459+0000 7f55d8ff9700 1 -- 172.21.15.23:0/821227821 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f55cc022070 con 0x7f55dc066c90 2024-03-13T21:32:31.527 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.536+0000 7f55e2138700 1 -- 172.21.15.23:0/821227821 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9ce90734-afdb-498c-952c-64a477a8f27a", "id": 2} v 0) v1 -- 0x7f55dc0a4850 con 0x7f55dc066c90 2024-03-13T21:32:31.533 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.543+0000 7f55d8ff9700 1 -- 172.21.15.23:0/821227821 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ce90734-afdb-498c-952c-64a477a8f27a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f55cc006990 con 0x7f55dc066c90 2024-03-13T21:32:31.534 INFO:teuthology.orchestra.run.smithi023.stdout:2 2024-03-13T21:32:31.535 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.544+0000 7f55e2138700 1 -- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f55dc066c90 msgr2=0x7f55dc092280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.535 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.544+0000 7f55e2138700 1 --2- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f55dc066c90 0x7f55dc092280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f55dc0666b0 tx=0x7f55cc015000).stop 2024-03-13T21:32:31.535 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.544+0000 7f55e2138700 1 -- 172.21.15.23:0/821227821 shutdown_connections 2024-03-13T21:32:31.535 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.544+0000 7f55e2138700 1 --2- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f55dc139f90 0x7f55dc149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.535 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.544+0000 7f55e2138700 1 --2- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f55dc066c90 0x7f55dc092280 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.535 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.544+0000 7f55e2138700 1 --2- 172.21.15.23:0/821227821 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f55dc139bb0 0x7f55dc148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.536 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.544+0000 7f55e2138700 1 -- 172.21.15.23:0/821227821 >> 172.21.15.23:0/821227821 conn(0x7f55dc0a89d0 msgr2=0x7f55dc136aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:31.536 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.544+0000 7f55e2138700 1 -- 172.21.15.23:0/821227821 shutdown_connections 2024-03-13T21:32:31.536 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.544+0000 7f55e2138700 1 -- 172.21.15.23:0/821227821 wait complete. 2024-03-13T21:32:31.545 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --cluster ceph osd new fbfaabf4-47ea-40d2-8c62-3239f195bfe7 3 2024-03-13T21:32:31.653 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.663+0000 7f2c9c571700 1 Processor -- start 2024-03-13T21:32:31.654 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.663+0000 7f2c9c571700 1 -- start start 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9c571700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c94133240 0x7f2c94133620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9c571700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c9413b600 0x7f2c9413d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9c571700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c94132df0 con 0x7f2c940646b0 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9c571700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c940639f0 con 0x7f2c9413b600 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9c571700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c94133020 con 0x7f2c94133240 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9a30d700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c94133240 0x7f2c94133620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c99b0c700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f2c940646b0 0x7f2c94136890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:49862/0 (socket says 172.21.15.23:49862) 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c99b0c700 1 -- 172.21.15.23:0/27476975 learned_addr learned my addr 172.21.15.23:0/27476975 (peer_addr_for_me v1:172.21.15.23:0/0) 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9ab0e700 1 --2- 172.21.15.23:0/27476975 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c9413b600 0x7f2c9413d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9930b700 1 -- 172.21.15.23:0/27476975 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427185683 0 0) 0x7f2c94132df0 con 0x7f2c940646b0 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9930b700 1 -- 172.21.15.23:0/27476975 --> v1:172.21.15.23:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c7c003260 con 0x7f2c940646b0 2024-03-13T21:32:31.656 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9ab0e700 1 -- 172.21.15.23:0/27476975 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c94133240 msgr2=0x7f2c94133620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:31.657 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9ab0e700 1 --2- 172.21.15.23:0/27476975 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c94133240 0x7f2c94133620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.657 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9ab0e700 1 -- 172.21.15.23:0/27476975 >> v1:172.21.15.23:6789/0 conn(0x7f2c940646b0 legacy=0x7f2c94136890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.657 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9ab0e700 1 -- 172.21.15.23:0/27476975 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c94136fa0 con 0x7f2c9413b600 2024-03-13T21:32:31.657 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.664+0000 7f2c9a30d700 1 --2- 172.21.15.23:0/27476975 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c94133240 0x7f2c94133620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:31.657 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.665+0000 7f2c9ab0e700 1 --2- 172.21.15.23:0/27476975 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c9413b600 0x7f2c9413d9f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f2c8c0032f0 tx=0x7f2c8c0091d0).ready entity=mon.1 client_cookie=b44b326c988cdefd server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:31.657 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.665+0000 7f2c9930b700 1 -- 172.21.15.23:0/27476975 <== mon.1 v2:172.21.15.23:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c8c00b640 con 0x7f2c9413b600 2024-03-13T21:32:31.657 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.665+0000 7f2c9930b700 1 -- 172.21.15.23:0/27476975 <== mon.1 v2:172.21.15.23:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c00b7a0 con 0x7f2c9413b600 2024-03-13T21:32:31.657 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.665+0000 7f2c9930b700 1 -- 172.21.15.23:0/27476975 <== mon.1 v2:172.21.15.23:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c8c00b910 con 0x7f2c9413b600 2024-03-13T21:32:31.660 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.666+0000 7f2c9c571700 1 -- 172.21.15.23:0/27476975 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c9413b600 msgr2=0x7f2c9413d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.660 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.666+0000 7f2c9c571700 1 --2- 172.21.15.23:0/27476975 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c9413b600 0x7f2c9413d9f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f2c8c0032f0 tx=0x7f2c8c0091d0).stop 2024-03-13T21:32:31.660 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.666+0000 7f2c9c571700 1 -- 172.21.15.23:0/27476975 shutdown_connections 2024-03-13T21:32:31.660 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.666+0000 7f2c9c571700 1 --2- 172.21.15.23:0/27476975 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c9413b600 0x7f2c9413d9f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.660 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.666+0000 7f2c9c571700 1 --2- 172.21.15.23:0/27476975 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c94133240 0x7f2c94133620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.660 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.666+0000 7f2c9c571700 1 -- 172.21.15.23:0/27476975 >> 172.21.15.23:0/27476975 conn(0x7f2c9405cf10 msgr2=0x7f2c9405d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.666+0000 7f2c9c571700 1 -- 172.21.15.23:0/27476975 shutdown_connections 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.666+0000 7f2c9c571700 1 -- 172.21.15.23:0/27476975 wait complete. 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.667+0000 7f2c9c571700 1 Processor -- start 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.667+0000 7f2c9c571700 1 -- start start 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9c571700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2c940646b0 0x7f2c94139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9c571700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c94133240 0x7f2c94146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9c571700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c9413b600 0x7f2c94146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9c571700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c940925c0 con 0x7f2c940646b0 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9c571700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9413a2e0 con 0x7f2c94133240 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9c571700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c94138d40 con 0x7f2c9413b600 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9a30d700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2c940646b0 0x7f2c94139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9a30d700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2c940646b0 0x7f2c94139910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:44188/0 (socket says 172.21.15.23:44188) 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9a30d700 1 -- 172.21.15.23:0/78617734 learned_addr learned my addr 172.21.15.23:0/78617734 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:31.661 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c99b0c700 1 --2- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c94133240 0x7f2c94146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.662 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9ab0e700 1 --2- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c9413b600 0x7f2c94146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.662 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9a30d700 1 -- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c9413b600 msgr2=0x7f2c94146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.662 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9a30d700 1 --2- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c9413b600 0x7f2c94146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.662 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9a30d700 1 -- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c94133240 msgr2=0x7f2c94146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.662 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9a30d700 1 --2- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c94133240 0x7f2c94146ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.662 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9a30d700 1 -- 172.21.15.23:0/78617734 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9000b180 con 0x7f2c940646b0 2024-03-13T21:32:31.662 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c99b0c700 1 --2- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c94133240 0x7f2c94146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:31.662 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.668+0000 7f2c9a30d700 1 --2- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2c940646b0 0x7f2c94139910 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f2c9000c000 tx=0x7f2c90006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:31.662 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.669+0000 7f2c837fe700 1 -- 172.21.15.23:0/78617734 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c9000c7e0 con 0x7f2c940646b0 2024-03-13T21:32:31.662 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.669+0000 7f2c837fe700 1 -- 172.21.15.23:0/78617734 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9000c940 con 0x7f2c940646b0 2024-03-13T21:32:31.664 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.669+0000 7f2c837fe700 1 -- 172.21.15.23:0/78617734 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c9000cca0 con 0x7f2c940646b0 2024-03-13T21:32:31.664 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.669+0000 7f2c9c571700 1 -- 172.21.15.23:0/78617734 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8c009470 con 0x7f2c940646b0 2024-03-13T21:32:31.664 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.669+0000 7f2c9c571700 1 -- 172.21.15.23:0/78617734 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c94131f50 con 0x7f2c940646b0 2024-03-13T21:32:31.664 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.669+0000 7f2c837fe700 1 -- 172.21.15.23:0/78617734 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f2c9001c460 con 0x7f2c940646b0 2024-03-13T21:32:31.664 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.669+0000 7f2c837fe700 1 -- 172.21.15.23:0/78617734 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f2c9001c6f0 con 0x7f2c940646b0 2024-03-13T21:32:31.665 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.670+0000 7f2c9c571700 1 -- 172.21.15.23:0/78617734 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c941362d0 con 0x7f2c940646b0 2024-03-13T21:32:31.665 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.674+0000 7f2c837fe700 1 -- 172.21.15.23:0/78617734 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f2c9001cb50 con 0x7f2c940646b0 2024-03-13T21:32:31.741 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.750+0000 7f2c9c571700 1 -- 172.21.15.23:0/78617734 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fbfaabf4-47ea-40d2-8c62-3239f195bfe7", "id": 3} v 0) v1 -- 0x7f2c94057a60 con 0x7f2c940646b0 2024-03-13T21:32:31.747 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.756+0000 7f2c837fe700 1 -- 172.21.15.23:0/78617734 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fbfaabf4-47ea-40d2-8c62-3239f195bfe7", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2c9001cd70 con 0x7f2c940646b0 2024-03-13T21:32:31.747 INFO:teuthology.orchestra.run.smithi023.stdout:3 2024-03-13T21:32:31.748 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.757+0000 7f2c9c571700 1 -- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2c940646b0 msgr2=0x7f2c94139910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.748 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.757+0000 7f2c9c571700 1 --2- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2c940646b0 0x7f2c94139910 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f2c9000c000 tx=0x7f2c90006df0).stop 2024-03-13T21:32:31.748 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.757+0000 7f2c9c571700 1 -- 172.21.15.23:0/78617734 shutdown_connections 2024-03-13T21:32:31.748 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.757+0000 7f2c9c571700 1 --2- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f2c9413b600 0x7f2c94146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.748 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.757+0000 7f2c9c571700 1 --2- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f2c940646b0 0x7f2c94139910 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.748 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.757+0000 7f2c9c571700 1 --2- 172.21.15.23:0/78617734 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f2c94133240 0x7f2c94146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.748 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.757+0000 7f2c9c571700 1 -- 172.21.15.23:0/78617734 >> 172.21.15.23:0/78617734 conn(0x7f2c9405cf10 msgr2=0x7f2c94099930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:31.748 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.757+0000 7f2c9c571700 1 -- 172.21.15.23:0/78617734 shutdown_connections 2024-03-13T21:32:31.749 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.757+0000 7f2c9c571700 1 -- 172.21.15.23:0/78617734 wait complete. 2024-03-13T21:32:31.759 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-13T21:32:31.759 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-13T21:32:31.761 INFO:tasks.ceph.osd.0:Started 2024-03-13T21:32:31.761 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-13T21:32:31.761 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-13T21:32:31.763 INFO:tasks.ceph.osd.1:Started 2024-03-13T21:32:31.763 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-13T21:32:31.763 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-13T21:32:31.765 INFO:tasks.ceph.osd.2:Started 2024-03-13T21:32:31.765 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-13T21:32:31.765 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-13T21:32:31.767 INFO:tasks.ceph.osd.3:Started 2024-03-13T21:32:31.768 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T21:32:31.922 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.931+0000 7f4bf76f9700 1 Processor -- start 2024-03-13T21:32:31.922 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.931+0000 7f4bf76f9700 1 -- start start 2024-03-13T21:32:31.924 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.932+0000 7f4bf76f9700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f4bf009b520 0x7f4bf009bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.924 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.932+0000 7f4bf76f9700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 0x7f4bf0099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.924 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.932+0000 7f4bf76f9700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf0136ef0 con 0x7f4bf01366f0 2024-03-13T21:32:31.924 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.932+0000 7f4bf76f9700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf009a380 con 0x7f4bf009b520 2024-03-13T21:32:31.924 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.932+0000 7f4bf76f9700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0135fe0 con 0x7f4bf0136ad0 2024-03-13T21:32:31.924 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bf5c96700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 0x7f4bf0099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bf5c96700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 0x7f4bf0099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:52572/0 (socket says 172.21.15.23:52572) 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bf5c96700 1 -- 172.21.15.23:0/2964866568 learned_addr learned my addr 172.21.15.23:0/2964866568 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bf4c94700 1 --2- 172.21.15.23:0/2964866568 >> v2:172.21.15.23:3300/0 conn(0x7f4bf009b520 0x7f4bf009bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bf5c96700 1 -- 172.21.15.23:0/2964866568 >> v2:172.21.15.23:3300/0 conn(0x7f4bf009b520 msgr2=0x7f4bf009bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bf5c96700 1 --2- 172.21.15.23:0/2964866568 >> v2:172.21.15.23:3300/0 conn(0x7f4bf009b520 0x7f4bf009bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bf5c96700 1 -- 172.21.15.23:0/2964866568 >> v1:172.21.15.23:6789/0 conn(0x7f4bf01366f0 legacy=0x7f4bf0139530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bf5c96700 1 -- 172.21.15.23:0/2964866568 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf009a0c0 con 0x7f4bf0136ad0 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bf5c96700 1 --2- 172.21.15.23:0/2964866568 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 0x7f4bf0099a00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4bf005c360 tx=0x7f4be8009290).ready entity=mon.2 client_cookie=9f9db36156632df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bdffff700 1 -- 172.21.15.23:0/2964866568 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be800b860 con 0x7f4bf0136ad0 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bdffff700 1 -- 172.21.15.23:0/2964866568 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be800b9c0 con 0x7f4bf0136ad0 2024-03-13T21:32:31.925 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.933+0000 7f4bdffff700 1 -- 172.21.15.23:0/2964866568 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4be800bb30 con 0x7f4bf0136ad0 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.934+0000 7f4bf76f9700 1 -- 172.21.15.23:0/2964866568 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 msgr2=0x7f4bf0099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.934+0000 7f4bf76f9700 1 --2- 172.21.15.23:0/2964866568 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 0x7f4bf0099a00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4bf005c360 tx=0x7f4be8009290).stop 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.934+0000 7f4bf76f9700 1 -- 172.21.15.23:0/2964866568 shutdown_connections 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.934+0000 7f4bf76f9700 1 --2- 172.21.15.23:0/2964866568 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 0x7f4bf0099a00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.934+0000 7f4bf76f9700 1 --2- 172.21.15.23:0/2964866568 >> v2:172.21.15.23:3300/0 conn(0x7f4bf009b520 0x7f4bf009bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.934+0000 7f4bf76f9700 1 -- 172.21.15.23:0/2964866568 >> 172.21.15.23:0/2964866568 conn(0x7f4bf006c0c0 msgr2=0x7f4bf006c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.934+0000 7f4bf76f9700 1 -- 172.21.15.23:0/2964866568 shutdown_connections 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.934+0000 7f4bf76f9700 1 -- 172.21.15.23:0/2964866568 wait complete. 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.935+0000 7f4bf76f9700 1 Processor -- start 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.935+0000 7f4bf76f9700 1 -- start start 2024-03-13T21:32:31.927 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf76f9700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4bf009b520 0x7f4bf0097730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.928 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf76f9700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f4bf01366f0 0x7f4bf0098320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.928 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf76f9700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 0x7f4bf0092fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:31.928 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf76f9700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0097030 con 0x7f4bf009b520 2024-03-13T21:32:31.928 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf76f9700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0094ff0 con 0x7f4bf01366f0 2024-03-13T21:32:31.928 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf76f9700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0097230 con 0x7f4bf0136ad0 2024-03-13T21:32:31.928 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf5495700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4bf009b520 0x7f4bf0097730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.928 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf5495700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4bf009b520 0x7f4bf0097730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:44246/0 (socket says 172.21.15.23:44246) 2024-03-13T21:32:31.928 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf5495700 1 -- 172.21.15.23:0/242776019 learned_addr learned my addr 172.21.15.23:0/242776019 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:31.929 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf5c96700 1 --2- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 0x7f4bf0092fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.929 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf4c94700 1 --2- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f4bf01366f0 0x7f4bf0098320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:31.929 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf5495700 1 -- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 msgr2=0x7f4bf0092fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.929 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf5495700 1 --2- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 0x7f4bf0092fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.929 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf5495700 1 -- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f4bf01366f0 msgr2=0x7f4bf0098320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.929 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf5495700 1 --2- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f4bf01366f0 0x7f4bf0098320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.930 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.936+0000 7f4bf5495700 1 -- 172.21.15.23:0/242776019 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8009530 con 0x7f4bf009b520 2024-03-13T21:32:31.930 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.937+0000 7f4bf5495700 1 --2- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4bf009b520 0x7f4bf0097730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f4bf005cde0 tx=0x7f4bec007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:31.930 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.937+0000 7f4bde7fc700 1 -- 172.21.15.23:0/242776019 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bec012400 con 0x7f4bf009b520 2024-03-13T21:32:31.930 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.937+0000 7f4bde7fc700 1 -- 172.21.15.23:0/242776019 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec012560 con 0x7f4bf009b520 2024-03-13T21:32:31.930 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.937+0000 7f4bf76f9700 1 -- 172.21.15.23:0/242776019 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf0097d00 con 0x7f4bf009b520 2024-03-13T21:32:31.930 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.937+0000 7f4bde7fc700 1 -- 172.21.15.23:0/242776019 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bec0128c0 con 0x7f4bf009b520 2024-03-13T21:32:31.930 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.937+0000 7f4bde7fc700 1 -- 172.21.15.23:0/242776019 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f4bec010070 con 0x7f4bf009b520 2024-03-13T21:32:31.930 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.937+0000 7f4bf76f9700 1 -- 172.21.15.23:0/242776019 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf00967c0 con 0x7f4bf009b520 2024-03-13T21:32:31.930 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.938+0000 7f4bde7fc700 1 -- 172.21.15.23:0/242776019 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f4bec01b8f0 con 0x7f4bf009b520 2024-03-13T21:32:31.930 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.938+0000 7f4bf76f9700 1 -- 172.21.15.23:0/242776019 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf0057a60 con 0x7f4bf009b520 2024-03-13T21:32:31.931 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:31.940+0000 7f4bde7fc700 1 -- 172.21.15.23:0/242776019 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f4bec017070 con 0x7f4bf009b520 2024-03-13T21:32:31.995 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.004+0000 7f4bf76f9700 1 -- 172.21.15.23:0/242776019 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bf0060880 con 0x7f4bf009b520 2024-03-13T21:32:31.995 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.004+0000 7f4bde7fc700 1 -- 172.21.15.23:0/242776019 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+3913 (secure 0 0 0) 0x7f4bec01b3e0 con 0x7f4bf009b520 2024-03-13T21:32:31.995 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:31.995 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":6,"fsid":"19813cba-5ddc-4ec3-85b1-5e37baeea636","created":"2024-03-13T21:32:30.539980+0000","modified":"2024-03-13T21:32:31.752174+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"a63e0271-0bb4-4f3b-b8a0-681e5b8ce56b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a721ee5e-d9d6-420e-ba1f-78ff60306e44","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9ce90734-afdb-498c-952c-64a477a8f27a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fbfaabf4-47ea-40d2-8c62-3239f195bfe7","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-03-13T21:32:31.996 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.005+0000 7f4bf76f9700 1 -- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4bf009b520 msgr2=0x7f4bf0097730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:31.996 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.005+0000 7f4bf76f9700 1 --2- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4bf009b520 0x7f4bf0097730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f4bf005cde0 tx=0x7f4bec007c50).stop 2024-03-13T21:32:31.996 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.005+0000 7f4bf76f9700 1 -- 172.21.15.23:0/242776019 shutdown_connections 2024-03-13T21:32:31.996 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.005+0000 7f4bf76f9700 1 --2- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f4bf0136ad0 0x7f4bf0092fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.996 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.005+0000 7f4bf76f9700 1 --2- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4bf009b520 0x7f4bf0097730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.997 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.005+0000 7f4bf76f9700 1 --2- 172.21.15.23:0/242776019 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f4bf01366f0 0x7f4bf0098320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:31.997 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.005+0000 7f4bf76f9700 1 -- 172.21.15.23:0/242776019 >> 172.21.15.23:0/242776019 conn(0x7f4bf006c0c0 msgr2=0x7f4bf0133310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:31.997 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.005+0000 7f4bf76f9700 1 -- 172.21.15.23:0/242776019 shutdown_connections 2024-03-13T21:32:31.997 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:32.005+0000 7f4bf76f9700 1 -- 172.21.15.23:0/242776019 wait complete. 2024-03-13T21:32:32.011 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-13T21:32:32.011 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-13T21:32:32.154 INFO:tasks.ceph.osd.1.smithi023.stderr:2024-03-13T21:32:32.163+0000 7f58057b3f00 -1 Falling back to public interface 2024-03-13T21:32:32.398 INFO:tasks.ceph.osd.0.smithi023.stderr:2024-03-13T21:32:32.407+0000 7f33d74b3f00 -1 Falling back to public interface 2024-03-13T21:32:32.401 INFO:tasks.ceph.osd.2.smithi023.stderr:2024-03-13T21:32:32.411+0000 7f1bc9777f00 -1 Falling back to public interface 2024-03-13T21:32:32.405 INFO:tasks.ceph.osd.3.smithi023.stderr:2024-03-13T21:32:32.414+0000 7fe55236ff00 -1 Falling back to public interface 2024-03-13T21:32:32.414 DEBUG:teuthology.orchestra.run.smithi023:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-13T21:32:32.519 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.528+0000 7f9a1ca71700 1 Processor -- start 2024-03-13T21:32:32.519 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.528+0000 7f9a1ca71700 1 -- start start 2024-03-13T21:32:32.520 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.529+0000 7f9a1ca71700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 0x7f9a180a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:32.520 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.529+0000 7f9a1ca71700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f9a180a1130 0x7f9a18136980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:32.520 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.529+0000 7f9a1ca71700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1809fea0 con 0x7f9a1809c4f0 2024-03-13T21:32:32.520 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.529+0000 7f9a1ca71700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a180a26c0 con 0x7f9a180a1130 2024-03-13T21:32:32.521 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.529+0000 7f9a1ca71700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180a2d00 con 0x7f9a181365a0 2024-03-13T21:32:32.521 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.529+0000 7f9a1659c700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 0x7f9a180a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:32.521 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.529+0000 7f9a15d9b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f9a180a1130 0x7f9a18136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:32.521 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.529+0000 7f9a15d9b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f9a180a1130 0x7f9a18136980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:44250/0 (socket says 172.21.15.23:44250) 2024-03-13T21:32:32.521 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.529+0000 7f9a1659c700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 0x7f9a180a20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:41180/0 (socket says 172.21.15.23:41180) 2024-03-13T21:32:32.521 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.529+0000 7f9a15d9b700 1 -- 172.21.15.23:0/1423433532 learned_addr learned my addr 172.21.15.23:0/1423433532 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:32.522 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.530+0000 7f9a15d9b700 1 -- 172.21.15.23:0/1423433532 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 msgr2=0x7f9a180a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:32.522 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.530+0000 7f9a15d9b700 1 --2- 172.21.15.23:0/1423433532 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 0x7f9a180a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:32.522 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.530+0000 7f9a15d9b700 1 -- 172.21.15.23:0/1423433532 >> v1:172.21.15.23:6789/0 conn(0x7f9a1809c4f0 legacy=0x7f9a180a3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:32.522 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.530+0000 7f9a15d9b700 1 -- 172.21.15.23:0/1423433532 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180a2e70 con 0x7f9a180a1130 2024-03-13T21:32:32.522 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.530+0000 7f9a15d9b700 1 --2- 172.21.15.23:0/1423433532 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a180a1130 0x7f9a18136980 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f9a0c002f70 tx=0x7f9a0c006f40).ready entity=mon.0 client_cookie=ce68101d6a57253d server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.530+0000 7f9a1559a700 1 -- 172.21.15.23:0/1423433532 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a0c00da80 con 0x7f9a180a1130 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.530+0000 7f9a1559a700 1 -- 172.21.15.23:0/1423433532 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c000e90 con 0x7f9a180a1130 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.530+0000 7f9a1559a700 1 -- 172.21.15.23:0/1423433532 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a0c00de10 con 0x7f9a180a1130 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.531+0000 7f9a1ca71700 1 -- 172.21.15.23:0/1423433532 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a180a1130 msgr2=0x7f9a18136980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.531+0000 7f9a1ca71700 1 --2- 172.21.15.23:0/1423433532 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a180a1130 0x7f9a18136980 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f9a0c002f70 tx=0x7f9a0c006f40).stop 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.531+0000 7f9a1ca71700 1 -- 172.21.15.23:0/1423433532 shutdown_connections 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.531+0000 7f9a1ca71700 1 --2- 172.21.15.23:0/1423433532 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 0x7f9a180a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.531+0000 7f9a1ca71700 1 --2- 172.21.15.23:0/1423433532 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a180a1130 0x7f9a18136980 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.531+0000 7f9a1ca71700 1 -- 172.21.15.23:0/1423433532 >> 172.21.15.23:0/1423433532 conn(0x7f9a180a8960 msgr2=0x7f9a180a8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.531+0000 7f9a1ca71700 1 -- 172.21.15.23:0/1423433532 shutdown_connections 2024-03-13T21:32:32.523 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.531+0000 7f9a1ca71700 1 -- 172.21.15.23:0/1423433532 wait complete. 2024-03-13T21:32:32.524 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.533+0000 7f9a1ca71700 1 Processor -- start 2024-03-13T21:32:32.524 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.533+0000 7f9a1ca71700 1 -- start start 2024-03-13T21:32:32.525 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.533+0000 7f9a1ca71700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a1809c4f0 0x7f9a1809dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:32.525 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.533+0000 7f9a1ca71700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a180a1130 0x7f9a1809c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:32.525 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1ca71700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 0x7f9a18145f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1ca71700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a18145d10 con 0x7f9a180a1130 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1ca71700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1809d3c0 con 0x7f9a181365a0 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1ca71700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1812e450 con 0x7f9a1809c4f0 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1659c700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a1809c4f0 0x7f9a1809dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1659c700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a1809c4f0 0x7f9a1809dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:52588/0 (socket says 172.21.15.23:52588) 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1659c700 1 -- 172.21.15.23:0/334608450 learned_addr learned my addr 172.21.15.23:0/334608450 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a16d9d700 1 --2- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 0x7f9a18145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a15d9b700 1 --2- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a180a1130 0x7f9a1809c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1659c700 1 -- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 msgr2=0x7f9a18145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1659c700 1 --2- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 0x7f9a18145f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1659c700 1 -- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a180a1130 msgr2=0x7f9a1809c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:32.526 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1659c700 1 --2- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a180a1130 0x7f9a1809c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:32.527 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1659c700 1 -- 172.21.15.23:0/334608450 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c00b460 con 0x7f9a1809c4f0 2024-03-13T21:32:32.527 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a16d9d700 1 --2- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 0x7f9a18145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-13T21:32:32.527 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.534+0000 7f9a1659c700 1 --2- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a1809c4f0 0x7f9a1809dd80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f9a0800c000 tx=0x7f9a08006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:32.527 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.535+0000 7f9a077fe700 1 -- 172.21.15.23:0/334608450 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a08010780 con 0x7f9a1809c4f0 2024-03-13T21:32:32.527 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.535+0000 7f9a077fe700 1 -- 172.21.15.23:0/334608450 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a080108e0 con 0x7f9a1809c4f0 2024-03-13T21:32:32.527 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.535+0000 7f9a1ca71700 1 -- 172.21.15.23:0/334608450 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1809ce50 con 0x7f9a1809c4f0 2024-03-13T21:32:32.527 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.535+0000 7f9a077fe700 1 -- 172.21.15.23:0/334608450 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a08010c40 con 0x7f9a1809c4f0 2024-03-13T21:32:32.528 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.535+0000 7f9a1ca71700 1 -- 172.21.15.23:0/334608450 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1809d6a0 con 0x7f9a1809c4f0 2024-03-13T21:32:32.528 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.536+0000 7f9a077fe700 1 -- 172.21.15.23:0/334608450 <== mon.2 v2:172.21.15.23:3302/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f9a0801c460 con 0x7f9a1809c4f0 2024-03-13T21:32:32.528 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.536+0000 7f9a077fe700 1 -- 172.21.15.23:0/334608450 <== mon.2 v2:172.21.15.23:3302/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f9a0801cc20 con 0x7f9a1809c4f0 2024-03-13T21:32:32.528 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.536+0000 7f9a1ca71700 1 -- 172.21.15.23:0/334608450 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a18130130 con 0x7f9a1809c4f0 2024-03-13T21:32:32.530 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.539+0000 7f9a077fe700 1 -- 172.21.15.23:0/334608450 <== mon.2 v2:172.21.15.23:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f9a0801ce30 con 0x7f9a1809c4f0 2024-03-13T21:32:32.599 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.607+0000 7f9a1ca71700 1 -- 172.21.15.23:0/334608450 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a1812ee90 con 0x7f9a1809c4f0 2024-03-13T21:32:32.599 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.608+0000 7f9a077fe700 1 -- 172.21.15.23:0/334608450 <== mon.2 v2:172.21.15.23:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+3913 (secure 0 0 0) 0x7f9a08026430 con 0x7f9a1809c4f0 2024-03-13T21:32:32.599 INFO:teuthology.misc.health.smithi023.stdout: 2024-03-13T21:32:32.600 INFO:teuthology.misc.health.smithi023.stdout:{"epoch":6,"fsid":"19813cba-5ddc-4ec3-85b1-5e37baeea636","created":"2024-03-13T21:32:30.539980+0000","modified":"2024-03-13T21:32:31.752174+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"a63e0271-0bb4-4f3b-b8a0-681e5b8ce56b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a721ee5e-d9d6-420e-ba1f-78ff60306e44","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9ce90734-afdb-498c-952c-64a477a8f27a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fbfaabf4-47ea-40d2-8c62-3239f195bfe7","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-03-13T21:32:32.600 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.609+0000 7f9a1ca71700 1 -- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a1809c4f0 msgr2=0x7f9a1809dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:32.601 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.609+0000 7f9a1ca71700 1 --2- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a1809c4f0 0x7f9a1809dd80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f9a0800c000 tx=0x7f9a08006df0).stop 2024-03-13T21:32:32.601 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.609+0000 7f9a1ca71700 1 -- 172.21.15.23:0/334608450 shutdown_connections 2024-03-13T21:32:32.601 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.609+0000 7f9a1ca71700 1 --2- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a181365a0 0x7f9a18145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:32.601 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.609+0000 7f9a1ca71700 1 --2- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a1809c4f0 0x7f9a1809dd80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:32.601 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.609+0000 7f9a1ca71700 1 --2- 172.21.15.23:0/334608450 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a180a1130 0x7f9a1809c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:32.601 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.609+0000 7f9a1ca71700 1 -- 172.21.15.23:0/334608450 >> 172.21.15.23:0/334608450 conn(0x7f9a180a8960 msgr2=0x7f9a181354e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:32.601 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.609+0000 7f9a1ca71700 1 -- 172.21.15.23:0/334608450 shutdown_connections 2024-03-13T21:32:32.601 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:32.609+0000 7f9a1ca71700 1 -- 172.21.15.23:0/334608450 wait complete. 2024-03-13T21:32:32.615 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-03-13T21:32:33.247 INFO:tasks.ceph.osd.1.smithi023.stderr:2024-03-13T21:32:33.255+0000 7f58057b3f00 -1 osd.1 0 log_to_monitors {default=true} 2024-03-13T21:32:33.490 INFO:tasks.ceph.osd.0.smithi023.stderr:2024-03-13T21:32:33.499+0000 7f33d74b3f00 -1 osd.0 0 log_to_monitors {default=true} 2024-03-13T21:32:33.491 INFO:tasks.ceph.osd.3.smithi023.stderr:2024-03-13T21:32:33.500+0000 7fe55236ff00 -1 osd.3 0 log_to_monitors {default=true} 2024-03-13T21:32:33.720 INFO:tasks.ceph.osd.2.smithi023.stderr:2024-03-13T21:32:33.730+0000 7f1bc9777f00 -1 osd.2 0 log_to_monitors {default=true} 2024-03-13T21:32:35.270 INFO:tasks.ceph.osd.1.smithi023.stderr:2024-03-13T21:32:35.279+0000 7f57edc7e700 -1 osd.1 0 waiting for initial osdmap 2024-03-13T21:32:35.270 INFO:tasks.ceph.osd.2.smithi023.stderr:2024-03-13T21:32:35.279+0000 7f1bb1c42700 -1 osd.2 0 waiting for initial osdmap 2024-03-13T21:32:35.284 INFO:tasks.ceph.osd.3.smithi023.stderr:2024-03-13T21:32:35.294+0000 7fe53a83a700 -1 osd.3 0 waiting for initial osdmap 2024-03-13T21:32:35.289 INFO:tasks.ceph.osd.0.smithi023.stderr:2024-03-13T21:32:35.298+0000 7f33bf97e700 -1 osd.0 0 waiting for initial osdmap 2024-03-13T21:32:35.290 INFO:tasks.ceph.osd.2.smithi023.stderr:2024-03-13T21:32:35.299+0000 7f1bbd65f700 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T21:32:35.291 INFO:tasks.ceph.osd.1.smithi023.stderr:2024-03-13T21:32:35.299+0000 7f57f969b700 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T21:32:35.294 INFO:tasks.ceph.osd.3.smithi023.stderr:2024-03-13T21:32:35.304+0000 7fe546257700 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T21:32:35.297 INFO:tasks.ceph.osd.0.smithi023.stderr:2024-03-13T21:32:35.307+0000 7f33cb39b700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T21:32:39.018 DEBUG:teuthology.orchestra.run.smithi023:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-13T21:32:39.129 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.138+0000 7f9ebadb3700 1 Processor -- start 2024-03-13T21:32:39.129 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.138+0000 7f9ebadb3700 1 -- start start 2024-03-13T21:32:39.132 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9ebadb3700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f9eb406c7d0 0x7f9eb4135d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.132 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9ebadb3700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb405d0d0 0x7f9eb405d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.132 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9ebadb3700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb41392f0 0x7f9eb4139780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.132 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9ebadb3700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb405cc20 con 0x7f9eb406c7d0 2024-03-13T21:32:39.132 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9ebadb3700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb405c720 con 0x7f9eb405d0d0 2024-03-13T21:32:39.132 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9ebadb3700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb41362d0 con 0x7f9eb41392f0 2024-03-13T21:32:39.132 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb8b4f700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f9eb406c7d0 0x7f9eb4135d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.132 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb9350700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb41392f0 0x7f9eb4139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb3fff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb405d0d0 0x7f9eb405d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb8b4f700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f9eb406c7d0 0x7f9eb4135d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49748/0 (socket says 172.21.15.23:49748) 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb9350700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb41392f0 0x7f9eb4139780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36262/0 (socket says 172.21.15.23:36262) 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb3fff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb405d0d0 0x7f9eb405d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44672/0 (socket says 172.21.15.23:44672) 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb8b4f700 1 -- 172.21.15.23:0/1425727817 learned_addr learned my addr 172.21.15.23:0/1425727817 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb8b4f700 1 -- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb41392f0 msgr2=0x7f9eb4139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb8b4f700 1 --2- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb41392f0 0x7f9eb4139780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb8b4f700 1 -- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb405d0d0 msgr2=0x7f9eb405d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb8b4f700 1 --2- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb405d0d0 0x7f9eb405d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb8b4f700 1 -- 172.21.15.23:0/1425727817 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4136590 con 0x7f9eb406c7d0 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb3fff700 1 --2- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb405d0d0 0x7f9eb405d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb9350700 1 --2- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb41392f0 0x7f9eb4139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:39.133 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.139+0000 7f9eb8b4f700 1 --2- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb406c7d0 0x7f9eb4135d20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f9ea8002f70 tx=0x7f9ea8006eb0).ready entity=mon.0 client_cookie=1c49d7a2f6b11cc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9eb37fe700 1 -- 172.21.15.23:0/1425727817 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ea800d570 con 0x7f9eb406c7d0 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9eb37fe700 1 -- 172.21.15.23:0/1425727817 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea8000e90 con 0x7f9eb406c7d0 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9eb37fe700 1 -- 172.21.15.23:0/1425727817 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ea800d900 con 0x7f9eb406c7d0 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9ebadb3700 1 -- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb406c7d0 msgr2=0x7f9eb4135d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9ebadb3700 1 --2- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb406c7d0 0x7f9eb4135d20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f9ea8002f70 tx=0x7f9ea8006eb0).stop 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9ebadb3700 1 -- 172.21.15.23:0/1425727817 shutdown_connections 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9ebadb3700 1 --2- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb41392f0 0x7f9eb4139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9ebadb3700 1 --2- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb406c7d0 0x7f9eb4135d20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9ebadb3700 1 --2- 172.21.15.23:0/1425727817 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb405d0d0 0x7f9eb405d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9ebadb3700 1 -- 172.21.15.23:0/1425727817 >> 172.21.15.23:0/1425727817 conn(0x7f9eb4067ab0 msgr2=0x7f9eb406c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9ebadb3700 1 -- 172.21.15.23:0/1425727817 shutdown_connections 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.140+0000 7f9ebadb3700 1 -- 172.21.15.23:0/1425727817 wait complete. 2024-03-13T21:32:39.134 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.141+0000 7f9ebadb3700 1 Processor -- start 2024-03-13T21:32:39.135 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.141+0000 7f9ebadb3700 1 -- start start 2024-03-13T21:32:39.137 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9ebadb3700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb405d0d0 0x7f9eb4132d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.137 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9ebadb3700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb406c7d0 0x7f9eb40a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.137 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9ebadb3700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb41392f0 0x7f9eb40a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.137 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9ebadb3700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40a2930 con 0x7f9eb405d0d0 2024-03-13T21:32:39.137 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9ebadb3700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40a2b80 con 0x7f9eb41392f0 2024-03-13T21:32:39.137 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9ebadb3700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40a1cc0 con 0x7f9eb406c7d0 2024-03-13T21:32:39.137 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb8b4f700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb405d0d0 0x7f9eb4132d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.137 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb9350700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb41392f0 0x7f9eb40a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb3fff700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb406c7d0 0x7f9eb40a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb8b4f700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb405d0d0 0x7f9eb4132d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49760/0 (socket says 172.21.15.23:49760) 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb3fff700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb406c7d0 0x7f9eb40a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36268/0 (socket says 172.21.15.23:36268) 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb8b4f700 1 -- 172.21.15.23:0/558110408 learned_addr learned my addr 172.21.15.23:0/558110408 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb9350700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb41392f0 0x7f9eb40a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44688/0 (socket says 172.21.15.23:44688) 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb8b4f700 1 -- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb406c7d0 msgr2=0x7f9eb40a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb8b4f700 1 --2- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb406c7d0 0x7f9eb40a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb8b4f700 1 -- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb41392f0 msgr2=0x7f9eb40a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb8b4f700 1 --2- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb41392f0 0x7f9eb40a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb8b4f700 1 -- 172.21.15.23:0/558110408 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea000b180 con 0x7f9eb405d0d0 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb3fff700 1 --2- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb406c7d0 0x7f9eb40a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb9350700 1 --2- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb41392f0 0x7f9eb40a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:39.138 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.142+0000 7f9eb8b4f700 1 --2- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb405d0d0 0x7f9eb4132d10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9ea8002f70 tx=0x7f9ea8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:39.139 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.143+0000 7f9eb1ffb700 1 -- 172.21.15.23:0/558110408 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ea8006b60 con 0x7f9eb405d0d0 2024-03-13T21:32:39.139 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.143+0000 7f9eb1ffb700 1 -- 172.21.15.23:0/558110408 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea8011c30 con 0x7f9eb405d0d0 2024-03-13T21:32:39.139 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.143+0000 7f9ebadb3700 1 -- 172.21.15.23:0/558110408 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac009470 con 0x7f9eb405d0d0 2024-03-13T21:32:39.139 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.143+0000 7f9eb1ffb700 1 -- 172.21.15.23:0/558110408 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ea801e460 con 0x7f9eb405d0d0 2024-03-13T21:32:39.139 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.143+0000 7f9ebadb3700 1 -- 172.21.15.23:0/558110408 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea800b1e0 con 0x7f9eb405d0d0 2024-03-13T21:32:39.139 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.144+0000 7f9eb1ffb700 1 -- 172.21.15.23:0/558110408 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f9ea8025030 con 0x7f9eb405d0d0 2024-03-13T21:32:39.139 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.144+0000 7f9ebadb3700 1 -- 172.21.15.23:0/558110408 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb40a5e10 con 0x7f9eb405d0d0 2024-03-13T21:32:39.139 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.144+0000 7f9eb1ffb700 1 -- 172.21.15.23:0/558110408 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2469+0+0 (secure 0 0 0) 0x7f9ea804b440 con 0x7f9eb405d0d0 2024-03-13T21:32:39.139 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.147+0000 7f9eb1ffb700 1 -- 172.21.15.23:0/558110408 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f9eb40a5e10 con 0x7f9eb405d0d0 2024-03-13T21:32:39.255 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.264+0000 7f9ebadb3700 1 -- 172.21.15.23:0/558110408 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9eb40a5e10 con 0x7f9eb405d0d0 2024-03-13T21:32:39.255 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.265+0000 7f9eb1ffb700 1 -- 172.21.15.23:0/558110408 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+6237 (secure 0 0 0) 0x7f9eb40a5e10 con 0x7f9eb405d0d0 2024-03-13T21:32:39.256 INFO:teuthology.misc.health.smithi023.stdout: 2024-03-13T21:32:39.256 INFO:teuthology.misc.health.smithi023.stdout:{"epoch":11,"fsid":"19813cba-5ddc-4ec3-85b1-5e37baeea636","created":"2024-03-13T21:32:30.539980+0000","modified":"2024-03-13T21:32:38.326712+0000","last_up_change":"2024-03-13T21:32:36.292795+0000","last_in_change":"2024-03-13T21:32:36.292795+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-03-13T21:32:33.319887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"a63e0271-0bb4-4f3b-b8a0-681e5b8ce56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6804","nonce":39974}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6805","nonce":39974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6807","nonce":39974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6806","nonce":39974}]},"public_addr":"172.21.15.23:6804/39974","cluster_addr":"172.21.15.23:6805/39974","heartbeat_back_addr":"172.21.15.23:6807/39974","heartbeat_front_addr":"172.21.15.23:6806/39974","state":["exists","up"]},{"osd":1,"uuid":"a721ee5e-d9d6-420e-ba1f-78ff60306e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6800","nonce":39975}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6801","nonce":39975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6803","nonce":39975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6802","nonce":39975}]},"public_addr":"172.21.15.23:6800/39975","cluster_addr":"172.21.15.23:6801/39975","heartbeat_back_addr":"172.21.15.23:6803/39975","heartbeat_front_addr":"172.21.15.23:6802/39975","state":["exists","up"]},{"osd":2,"uuid":"9ce90734-afdb-498c-952c-64a477a8f27a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6808","nonce":39977}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6809","nonce":39977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6811","nonce":39977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6810","nonce":39977}]},"public_addr":"172.21.15.23:6808/39977","cluster_addr":"172.21.15.23:6809/39977","heartbeat_back_addr":"172.21.15.23:6811/39977","heartbeat_front_addr":"172.21.15.23:6810/39977","state":["exists","up"]},{"osd":3,"uuid":"fbfaabf4-47ea-40d2-8c62-3239f195bfe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6812","nonce":39976}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6813","nonce":39976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6815","nonce":39976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6814","nonce":39976}]},"public_addr":"172.21.15.23:6812/39976","cluster_addr":"172.21.15.23:6813/39976","heartbeat_back_addr":"172.21.15.23:6815/39976","heartbeat_front_addr":"172.21.15.23:6814/39976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.474369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.254337+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.761561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.481537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-03-13T21:32:39.257 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.266+0000 7f9ebadb3700 1 -- 172.21.15.23:0/558110408 >> v1:172.21.15.23:6816/39358 conn(0x7f9e94038dd0 legacy=0x7f9e9403b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:39.257 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.266+0000 7f9ebadb3700 1 -- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb405d0d0 msgr2=0x7f9eb4132d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:39.257 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.266+0000 7f9ebadb3700 1 --2- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb405d0d0 0x7f9eb4132d10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9ea8002f70 tx=0x7f9ea8015000).stop 2024-03-13T21:32:39.258 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.266+0000 7f9ebadb3700 1 -- 172.21.15.23:0/558110408 shutdown_connections 2024-03-13T21:32:39.258 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.266+0000 7f9ebadb3700 1 --2- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9eb41392f0 0x7f9eb40a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.258 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.266+0000 7f9ebadb3700 1 --2- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9eb405d0d0 0x7f9eb4132d10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.258 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.266+0000 7f9ebadb3700 1 --2- 172.21.15.23:0/558110408 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9eb406c7d0 0x7f9eb40a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.258 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.266+0000 7f9ebadb3700 1 -- 172.21.15.23:0/558110408 >> 172.21.15.23:0/558110408 conn(0x7f9eb4067ab0 msgr2=0x7f9eb40a9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:39.258 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.266+0000 7f9ebadb3700 1 -- 172.21.15.23:0/558110408 shutdown_connections 2024-03-13T21:32:39.258 INFO:teuthology.misc.health.smithi023.stderr:2024-03-13T21:32:39.266+0000 7f9ebadb3700 1 -- 172.21.15.23:0/558110408 wait complete. 2024-03-13T21:32:39.267 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-03-13T21:32:39.268 INFO:tasks.ceph:Creating RBD pool 2024-03-13T21:32:39.268 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --cluster ceph osd pool create rbd 8 2024-03-13T21:32:39.374 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.383+0000 7f3c3ceea700 1 Processor -- start 2024-03-13T21:32:39.374 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.383+0000 7f3c3ceea700 1 -- start start 2024-03-13T21:32:39.377 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c3ceea700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c3809aa10 0x7f3c380621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.377 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c3ceea700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f3c38064700 0x7f3c38136140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.377 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c3ceea700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809adf0 0x7f3c38132c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.377 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c3ceea700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3809b2a0 con 0x7f3c38064700 2024-03-13T21:32:39.377 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c3ceea700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3809bc50 con 0x7f3c3809aa10 2024-03-13T21:32:39.377 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c3ceea700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c381367e0 con 0x7f3c3809adf0 2024-03-13T21:32:39.377 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c35d9b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f3c38064700 0x7f3c38136140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c36d9d700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809adf0 0x7f3c38132c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c3659c700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c3809aa10 0x7f3c380621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c35d9b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f3c38064700 0x7f3c38136140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49768/0 (socket says 172.21.15.23:49768) 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c3659c700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c3809aa10 0x7f3c380621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44692/0 (socket says 172.21.15.23:44692) 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.384+0000 7f3c35d9b700 1 -- 172.21.15.23:0/3315079102 learned_addr learned my addr 172.21.15.23:0/3315079102 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c36d9d700 1 -- 172.21.15.23:0/3315079102 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c3809aa10 msgr2=0x7f3c380621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c36d9d700 1 --2- 172.21.15.23:0/3315079102 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c3809aa10 0x7f3c380621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c36d9d700 1 -- 172.21.15.23:0/3315079102 >> v2:172.21.15.23:3300/0 conn(0x7f3c38064700 msgr2=0x7f3c38136140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c36d9d700 1 --2- 172.21.15.23:0/3315079102 >> v2:172.21.15.23:3300/0 conn(0x7f3c38064700 0x7f3c38136140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c36d9d700 1 -- 172.21.15.23:0/3315079102 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c38136aa0 con 0x7f3c3809adf0 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c35d9b700 1 --2- 172.21.15.23:0/3315079102 >> v2:172.21.15.23:3300/0 conn(0x7f3c38064700 0x7f3c38136140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:39.378 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3659c700 1 --2- 172.21.15.23:0/3315079102 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c3809aa10 0x7f3c380621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c36d9d700 1 --2- 172.21.15.23:0/3315079102 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809adf0 0x7f3c38132c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f3c300032f0 tx=0x7f3c300091d0).ready entity=mon.2 client_cookie=d5c294de5c704f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3559a700 1 -- 172.21.15.23:0/3315079102 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c3000b600 con 0x7f3c3809adf0 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3559a700 1 -- 172.21.15.23:0/3315079102 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3000b760 con 0x7f3c3809adf0 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3559a700 1 -- 172.21.15.23:0/3315079102 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c3000b8d0 con 0x7f3c3809adf0 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3ceea700 1 -- 172.21.15.23:0/3315079102 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809adf0 msgr2=0x7f3c38132c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3ceea700 1 --2- 172.21.15.23:0/3315079102 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809adf0 0x7f3c38132c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f3c300032f0 tx=0x7f3c300091d0).stop 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3ceea700 1 -- 172.21.15.23:0/3315079102 shutdown_connections 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3ceea700 1 --2- 172.21.15.23:0/3315079102 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809adf0 0x7f3c38132c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3ceea700 1 --2- 172.21.15.23:0/3315079102 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c3809aa10 0x7f3c380621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3ceea700 1 --2- 172.21.15.23:0/3315079102 >> v2:172.21.15.23:3300/0 conn(0x7f3c38064700 0x7f3c38136140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3ceea700 1 -- 172.21.15.23:0/3315079102 >> 172.21.15.23:0/3315079102 conn(0x7f3c3806c0c0 msgr2=0x7f3c3806c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.385+0000 7f3c3ceea700 1 -- 172.21.15.23:0/3315079102 shutdown_connections 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.386+0000 7f3c3ceea700 1 -- 172.21.15.23:0/3315079102 wait complete. 2024-03-13T21:32:39.379 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.386+0000 7f3c3ceea700 1 Processor -- start 2024-03-13T21:32:39.380 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.386+0000 7f3c3ceea700 1 -- start start 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3ceea700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c38064700 0x7f3c38066f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3ceea700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809aa10 0x7f3c3809d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3ceea700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3c3809adf0 0x7f3c38097d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3ceea700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38098700 con 0x7f3c3809adf0 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3ceea700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38145930 con 0x7f3c38064700 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3ceea700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c38098e10 con 0x7f3c3809aa10 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3659c700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c38064700 0x7f3c38066f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c35d9b700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809aa10 0x7f3c3809d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3659c700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c38064700 0x7f3c38066f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44702/0 (socket says 172.21.15.23:44702) 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c36d9d700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3c3809adf0 0x7f3c38097d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:39.382 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3659c700 1 -- 172.21.15.23:0/1411053257 learned_addr learned my addr 172.21.15.23:0/1411053257 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3659c700 1 -- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809aa10 msgr2=0x7f3c3809d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3659c700 1 --2- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809aa10 0x7f3c3809d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3659c700 1 -- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3c3809adf0 msgr2=0x7f3c38097d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3659c700 1 --2- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3c3809adf0 0x7f3c38097d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c3659c700 1 -- 172.21.15.23:0/1411053257 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2800b1e0 con 0x7f3c38064700 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.387+0000 7f3c35d9b700 1 --2- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809aa10 0x7f3c3809d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.388+0000 7f3c3659c700 1 --2- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c38064700 0x7f3c38066f30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f3c2c00c000 tx=0x7f3c2c006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.388+0000 7f3c277fe700 1 -- 172.21.15.23:0/1411053257 <== mon.1 v2:172.21.15.23:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c2c00c7e0 con 0x7f3c38064700 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.388+0000 7f3c277fe700 1 -- 172.21.15.23:0/1411053257 <== mon.1 v2:172.21.15.23:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c00c940 con 0x7f3c38064700 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.388+0000 7f3c277fe700 1 -- 172.21.15.23:0/1411053257 <== mon.1 v2:172.21.15.23:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c2c00cca0 con 0x7f3c38064700 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.388+0000 7f3c3ceea700 1 -- 172.21.15.23:0/1411053257 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c00b180 con 0x7f3c38064700 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.388+0000 7f3c3ceea700 1 -- 172.21.15.23:0/1411053257 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c30009470 con 0x7f3c38064700 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.389+0000 7f3c277fe700 1 -- 172.21.15.23:0/1411053257 <== mon.1 v2:172.21.15.23:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f3c2c01c460 con 0x7f3c38064700 2024-03-13T21:32:39.383 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.389+0000 7f3c277fe700 1 -- 172.21.15.23:0/1411053257 <== mon.1 v2:172.21.15.23:3301/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2469+0+0 (secure 0 0 0) 0x7f3c2c04a8e0 con 0x7f3c38064700 2024-03-13T21:32:39.384 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.389+0000 7f3c3ceea700 1 -- 172.21.15.23:0/1411053257 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3809df00 con 0x7f3c38064700 2024-03-13T21:32:39.384 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.393+0000 7f3c277fe700 1 -- 172.21.15.23:0/1411053257 <== mon.1 v2:172.21.15.23:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f3c2c0336c0 con 0x7f3c38064700 2024-03-13T21:32:39.499 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:39.508+0000 7f3c3ceea700 1 -- 172.21.15.23:0/1411053257 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f3c38057a60 con 0x7f3c38064700 2024-03-13T21:32:40.340 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.349+0000 7f3c277fe700 1 -- 172.21.15.23:0/1411053257 <== mon.1 v2:172.21.15.23:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v12) v1 ==== 109+0+0 (secure 0 0 0) 0x7f3c2c02b0a0 con 0x7f3c38064700 2024-03-13T21:32:40.342 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.351+0000 7f3c3ceea700 1 -- 172.21.15.23:0/1411053257 >> v1:172.21.15.23:6816/39358 conn(0x7f3c14038dd0 legacy=0x7f3c1403b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:40.342 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.351+0000 7f3c3ceea700 1 -- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c38064700 msgr2=0x7f3c38066f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:40.342 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.351+0000 7f3c3ceea700 1 --2- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c38064700 0x7f3c38066f30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f3c2c00c000 tx=0x7f3c2c006df0).stop 2024-03-13T21:32:40.342 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.351+0000 7f3c3ceea700 1 -- 172.21.15.23:0/1411053257 shutdown_connections 2024-03-13T21:32:40.342 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.351+0000 7f3c3ceea700 1 --2- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3c3809adf0 0x7f3c38097d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:40.342 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.351+0000 7f3c3ceea700 1 --2- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3c38064700 0x7f3c38066f30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:40.342 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.351+0000 7f3c3ceea700 1 --2- 172.21.15.23:0/1411053257 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3c3809aa10 0x7f3c3809d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:40.342 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.351+0000 7f3c3ceea700 1 -- 172.21.15.23:0/1411053257 >> 172.21.15.23:0/1411053257 conn(0x7f3c3806c0c0 msgr2=0x7f3c380a7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:40.343 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.351+0000 7f3c3ceea700 1 -- 172.21.15.23:0/1411053257 shutdown_connections 2024-03-13T21:32:40.343 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.351+0000 7f3c3ceea700 1 -- 172.21.15.23:0/1411053257 wait complete. 2024-03-13T21:32:40.343 INFO:teuthology.orchestra.run.smithi023.stderr:pool 'rbd' created 2024-03-13T21:32:40.356 DEBUG:teuthology.orchestra.run.smithi023:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-03-13T21:32:40.468 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.477+0000 7f3ef244e700 1 Processor -- start 2024-03-13T21:32:40.469 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.477+0000 7f3ef244e700 1 -- start start 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.478+0000 7f3ef244e700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec09aa10 0x7f3eec0621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.478+0000 7f3ef244e700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 0x7f3eec09adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.478+0000 7f3ef244e700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec09bc50 con 0x7f3eec064700 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.478+0000 7f3ef244e700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec1365f0 con 0x7f3eec09aa10 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.478+0000 7f3ef244e700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec09b2e0 con 0x7f3eec136140 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.478+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec09aa10 0x7f3eec0621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.478+0000 7f3eeb7fe700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f3eec064700 0x7f3eec132c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59052/0 (socket says 172.21.15.23:59052) 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.478+0000 7f3eeb7fe700 1 -- 172.21.15.23:0/2751590863 learned_addr learned my addr 172.21.15.23:0/2751590863 (peer_addr_for_me v1:172.21.15.23:0/0) 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.478+0000 7f3ef09eb700 1 --2- 172.21.15.23:0/2751590863 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 0x7f3eec09adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3eeaffd700 1 -- 172.21.15.23:0/2751590863 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100497824 0 0) 0x7f3eec09bc50 con 0x7f3eec064700 2024-03-13T21:32:40.471 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3ef09eb700 1 -- 172.21.15.23:0/2751590863 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec09aa10 msgr2=0x7f3eec0621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3ef09eb700 1 --2- 172.21.15.23:0/2751590863 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec09aa10 0x7f3eec0621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3ef09eb700 1 -- 172.21.15.23:0/2751590863 >> v1:172.21.15.23:6789/0 conn(0x7f3eec064700 legacy=0x7f3eec132c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3ef09eb700 1 -- 172.21.15.23:0/2751590863 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec1367e0 con 0x7f3eec136140 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3eebfff700 1 --2- 172.21.15.23:0/2751590863 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec09aa10 0x7f3eec0621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3ef09eb700 1 --2- 172.21.15.23:0/2751590863 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 0x7f3eec09adf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f3ee40032f0 tx=0x7f3ee40091d0).ready entity=mon.2 client_cookie=576880a4be0afef4 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3eeaffd700 1 -- 172.21.15.23:0/2751590863 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ee400b6c0 con 0x7f3eec136140 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3eeaffd700 1 -- 172.21.15.23:0/2751590863 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee400b820 con 0x7f3eec136140 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3eeaffd700 1 -- 172.21.15.23:0/2751590863 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ee400b990 con 0x7f3eec136140 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3ef244e700 1 -- 172.21.15.23:0/2751590863 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 msgr2=0x7f3eec09adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3ef244e700 1 --2- 172.21.15.23:0/2751590863 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 0x7f3eec09adf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f3ee40032f0 tx=0x7f3ee40091d0).stop 2024-03-13T21:32:40.472 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.479+0000 7f3ef244e700 1 -- 172.21.15.23:0/2751590863 shutdown_connections 2024-03-13T21:32:40.473 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.480+0000 7f3ef244e700 1 --2- 172.21.15.23:0/2751590863 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 0x7f3eec09adf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:40.473 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.480+0000 7f3ef244e700 1 --2- 172.21.15.23:0/2751590863 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec09aa10 0x7f3eec0621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:40.473 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.480+0000 7f3ef244e700 1 -- 172.21.15.23:0/2751590863 >> 172.21.15.23:0/2751590863 conn(0x7f3eec06c0c0 msgr2=0x7f3eec06c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:40.473 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.480+0000 7f3ef244e700 1 -- 172.21.15.23:0/2751590863 shutdown_connections 2024-03-13T21:32:40.473 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.480+0000 7f3ef244e700 1 -- 172.21.15.23:0/2751590863 wait complete. 2024-03-13T21:32:40.475 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.480+0000 7f3ef244e700 1 Processor -- start 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.480+0000 7f3ef244e700 1 -- start start 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.481+0000 7f3ef244e700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec064700 0x7f3eec0989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.481+0000 7f3ef244e700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3eec09aa10 0x7f3eec097b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.481+0000 7f3ef244e700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 0x7f3eec148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.481+0000 7f3ef244e700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec148be0 con 0x7f3eec09aa10 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.481+0000 7f3ef244e700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec09da90 con 0x7f3eec064700 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.481+0000 7f3ef244e700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec09dc50 con 0x7f3eec136140 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.481+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec064700 0x7f3eec0989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.481+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec064700 0x7f3eec0989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44714/0 (socket says 172.21.15.23:44714) 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.481+0000 7f3ef09eb700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 0x7f3eec148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.481+0000 7f3eebfff700 1 -- 172.21.15.23:0/589196477 learned_addr learned my addr 172.21.15.23:0/589196477 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:40.476 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3eeb7fe700 1 --2- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3eec09aa10 0x7f3eec097b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3eebfff700 1 -- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 msgr2=0x7f3eec148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3eebfff700 1 --2- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 0x7f3eec148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3eebfff700 1 -- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3eec09aa10 msgr2=0x7f3eec097b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3eebfff700 1 --2- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3eec09aa10 0x7f3eec097b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3eebfff700 1 -- 172.21.15.23:0/589196477 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc00d410 con 0x7f3eec064700 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3eeb7fe700 1 --2- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3eec09aa10 0x7f3eec097b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3ef09eb700 1 --2- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 0x7f3eec148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3eebfff700 1 --2- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec064700 0x7f3eec0989e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f3edc00e000 tx=0x7f3edc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3ee97fa700 1 -- 172.21.15.23:0/589196477 <== mon.1 v2:172.21.15.23:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3edc00bbd0 con 0x7f3eec064700 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3ee97fa700 1 -- 172.21.15.23:0/589196477 <== mon.1 v2:172.21.15.23:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc00bd30 con 0x7f3eec064700 2024-03-13T21:32:40.477 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3ef244e700 1 -- 172.21.15.23:0/589196477 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee4009470 con 0x7f3eec064700 2024-03-13T21:32:40.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3ee97fa700 1 -- 172.21.15.23:0/589196477 <== mon.1 v2:172.21.15.23:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3edc00bea0 con 0x7f3eec064700 2024-03-13T21:32:40.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.482+0000 7f3ef244e700 1 -- 172.21.15.23:0/589196477 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec0971e0 con 0x7f3eec064700 2024-03-13T21:32:40.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.483+0000 7f3ee97fa700 1 -- 172.21.15.23:0/589196477 <== mon.1 v2:172.21.15.23:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f3edc010630 con 0x7f3eec064700 2024-03-13T21:32:40.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.483+0000 7f3ef244e700 1 -- 172.21.15.23:0/589196477 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec136aa0 con 0x7f3eec064700 2024-03-13T21:32:40.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.483+0000 7f3ee97fa700 1 -- 172.21.15.23:0/589196477 <== mon.1 v2:172.21.15.23:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2811+0+0 (secure 0 0 0) 0x7f3edc04b140 con 0x7f3eec064700 2024-03-13T21:32:40.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.486+0000 7f3ee97fa700 1 -- 172.21.15.23:0/589196477 <== mon.1 v2:172.21.15.23:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f3edc0307e0 con 0x7f3eec064700 2024-03-13T21:32:40.593 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:40.602+0000 7f3ef244e700 1 -- 172.21.15.23:0/589196477 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3eec057a60 con 0x7f3eec064700 2024-03-13T21:32:41.360 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.369+0000 7f3ee97fa700 1 -- 172.21.15.23:0/589196477 <== mon.1 v2:172.21.15.23:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v13) v1 ==== 173+0+0 (secure 0 0 0) 0x7f3edc033920 con 0x7f3eec064700 2024-03-13T21:32:41.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.371+0000 7f3ef244e700 1 -- 172.21.15.23:0/589196477 >> v1:172.21.15.23:6816/39358 conn(0x7f3ecc038e60 legacy=0x7f3ecc03b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.371+0000 7f3ef244e700 1 -- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec064700 msgr2=0x7f3eec0989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.371+0000 7f3ef244e700 1 --2- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec064700 0x7f3eec0989e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f3edc00e000 tx=0x7f3edc008040).stop 2024-03-13T21:32:41.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.371+0000 7f3ef244e700 1 -- 172.21.15.23:0/589196477 shutdown_connections 2024-03-13T21:32:41.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.371+0000 7f3ef244e700 1 --2- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f3eec136140 0x7f3eec148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.371+0000 7f3ef244e700 1 --2- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f3eec064700 0x7f3eec0989e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.371+0000 7f3ef244e700 1 --2- 172.21.15.23:0/589196477 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f3eec09aa10 0x7f3eec097b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.371+0000 7f3ef244e700 1 -- 172.21.15.23:0/589196477 >> 172.21.15.23:0/589196477 conn(0x7f3eec06c0c0 msgr2=0x7f3eec131310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:41.363 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.371+0000 7f3ef244e700 1 -- 172.21.15.23:0/589196477 shutdown_connections 2024-03-13T21:32:41.363 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.371+0000 7f3ef244e700 1 -- 172.21.15.23:0/589196477 wait complete. 2024-03-13T21:32:41.364 INFO:teuthology.orchestra.run.smithi023.stderr:enabled application 'rbd' on pool 'rbd' 2024-03-13T21:32:41.378 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-13T21:32:41.379 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-13T21:32:41.379 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-13T21:32:41.381 INFO:tasks.ceph.mds.a:Started 2024-03-13T21:32:41.381 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-13T21:32:41.381 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-13T21:32:41.383 INFO:tasks.ceph.mds.b:Started 2024-03-13T21:32:41.383 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-13T21:32:41.383 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-13T21:32:41.385 INFO:tasks.ceph.mds.c:Started 2024-03-13T21:32:41.385 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-13T21:32:41.386 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T21:32:41.467 INFO:tasks.ceph.mds.a.smithi023.stdout:starting mds.a at 2024-03-13T21:32:41.467 INFO:tasks.ceph.mds.b.smithi023.stdout:starting mds.b at 2024-03-13T21:32:41.471 INFO:tasks.ceph.mds.c.smithi023.stdout:starting mds.c at 2024-03-13T21:32:41.509 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.518+0000 7f0b9f6cc700 1 Processor -- start 2024-03-13T21:32:41.509 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.518+0000 7f0b9f6cc700 1 -- start start 2024-03-13T21:32:41.511 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9f6cc700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b981366f0 0x7f0b9809bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:41.512 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9f6cc700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f0b9809b520 0x7f0b98136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:41.512 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9f6cc700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b98099e70 con 0x7f0b98099a00 2024-03-13T21:32:41.512 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9f6cc700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9809a380 con 0x7f0b9809b520 2024-03-13T21:32:41.512 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9f6cc700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98135fe0 con 0x7f0b981366f0 2024-03-13T21:32:41.512 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9cc67700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f0b9809b520 0x7f0b98136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:41.512 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9d468700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b981366f0 0x7f0b9809bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:41.512 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9cc67700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f0b9809b520 0x7f0b98136ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49830/0 (socket says 172.21.15.23:49830) 2024-03-13T21:32:41.512 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9d468700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b981366f0 0x7f0b9809bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36320/0 (socket says 172.21.15.23:36320) 2024-03-13T21:32:41.512 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9cc67700 1 -- 172.21.15.23:0/3076036765 learned_addr learned my addr 172.21.15.23:0/3076036765 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:41.512 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9cc67700 1 -- 172.21.15.23:0/3076036765 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b981366f0 msgr2=0x7f0b9809bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9cc67700 1 --2- 172.21.15.23:0/3076036765 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b981366f0 0x7f0b9809bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9cc67700 1 -- 172.21.15.23:0/3076036765 >> v1:172.21.15.23:6789/0 conn(0x7f0b98099a00 legacy=0x7f0b98139530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9cc67700 1 -- 172.21.15.23:0/3076036765 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9809a0c0 con 0x7f0b9809b520 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b87fff700 1 -- 172.21.15.23:0/3076036765 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211331169 0 0) 0x7f0b98099e70 con 0x7f0b98099a00 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.519+0000 7f0b9d468700 1 --2- 172.21.15.23:0/3076036765 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b981366f0 0x7f0b9809bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b9cc67700 1 --2- 172.21.15.23:0/3076036765 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0b9809b520 0x7f0b98136ad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0b9805c360 tx=0x7f0b94006e50).ready entity=mon.0 client_cookie=87458fc3b2aeef4f server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b87fff700 1 -- 172.21.15.23:0/3076036765 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b9400d5c0 con 0x7f0b9809b520 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b87fff700 1 -- 172.21.15.23:0/3076036765 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b94000e90 con 0x7f0b9809b520 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b87fff700 1 -- 172.21.15.23:0/3076036765 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b9400d910 con 0x7f0b9809b520 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/3076036765 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0b9809b520 msgr2=0x7f0b98136ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b9f6cc700 1 --2- 172.21.15.23:0/3076036765 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0b9809b520 0x7f0b98136ad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0b9805c360 tx=0x7f0b94006e50).stop 2024-03-13T21:32:41.513 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/3076036765 shutdown_connections 2024-03-13T21:32:41.514 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b9f6cc700 1 --2- 172.21.15.23:0/3076036765 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b981366f0 0x7f0b9809bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.514 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b9f6cc700 1 --2- 172.21.15.23:0/3076036765 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0b9809b520 0x7f0b98136ad0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.514 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/3076036765 >> 172.21.15.23:0/3076036765 conn(0x7f0b9806c0c0 msgr2=0x7f0b9806c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:41.514 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/3076036765 shutdown_connections 2024-03-13T21:32:41.514 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.520+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/3076036765 wait complete. 2024-03-13T21:32:41.516 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.521+0000 7f0b9f6cc700 1 Processor -- start 2024-03-13T21:32:41.516 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.521+0000 7f0b9f6cc700 1 -- start start 2024-03-13T21:32:41.516 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9f6cc700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0b98099a00 0x7f0b98095510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:41.516 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9f6cc700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b9809b520 0x7f0b98148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:41.516 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9f6cc700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0b981366f0 0x7f0b981491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9f6cc700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980984f0 con 0x7f0b981366f0 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9f6cc700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980982d0 con 0x7f0b98099a00 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9f6cc700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98096b00 con 0x7f0b9809b520 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9cc67700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b9809b520 0x7f0b98148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9dc69700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0b981366f0 0x7f0b981491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9cc67700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b9809b520 0x7f0b98148ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36336/0 (socket says 172.21.15.23:36336) 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9cc67700 1 -- 172.21.15.23:0/1218724619 learned_addr learned my addr 172.21.15.23:0/1218724619 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9d468700 1 --2- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0b98099a00 0x7f0b98095510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9d468700 1 -- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b9809b520 msgr2=0x7f0b98148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9d468700 1 --2- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b9809b520 0x7f0b98148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.517 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9d468700 1 -- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0b981366f0 msgr2=0x7f0b981491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9d468700 1 --2- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0b981366f0 0x7f0b981491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9d468700 1 -- 172.21.15.23:0/1218724619 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8800b180 con 0x7f0b98099a00 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9cc67700 1 --2- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b9809b520 0x7f0b98148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9dc69700 1 --2- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0b981366f0 0x7f0b981491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b9d468700 1 --2- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0b98099a00 0x7f0b98095510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0b8800c000 tx=0x7f0b88006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.522+0000 7f0b867fc700 1 -- 172.21.15.23:0/1218724619 <== mon.1 v2:172.21.15.23:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b88010780 con 0x7f0b98099a00 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.523+0000 7f0b867fc700 1 -- 172.21.15.23:0/1218724619 <== mon.1 v2:172.21.15.23:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b880108e0 con 0x7f0b98099a00 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.523+0000 7f0b867fc700 1 -- 172.21.15.23:0/1218724619 <== mon.1 v2:172.21.15.23:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b88010c40 con 0x7f0b98099a00 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.523+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/1218724619 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9400b1e0 con 0x7f0b98099a00 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.523+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/1218724619 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b98095ee0 con 0x7f0b98099a00 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.523+0000 7f0b867fc700 1 -- 172.21.15.23:0/1218724619 <== mon.1 v2:172.21.15.23:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f0b8801c460 con 0x7f0b98099a00 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.524+0000 7f0b867fc700 1 -- 172.21.15.23:0/1218724619 <== mon.1 v2:172.21.15.23:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f0b8804a950 con 0x7f0b98099a00 2024-03-13T21:32:41.518 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.524+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/1218724619 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b980987c0 con 0x7f0b98099a00 2024-03-13T21:32:41.519 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.527+0000 7f0b867fc700 1 -- 172.21.15.23:0/1218724619 <== mon.1 v2:172.21.15.23:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f0b880335d0 con 0x7f0b98099a00 2024-03-13T21:32:41.635 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.644+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/1218724619 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b98097e40 con 0x7f0b98099a00 2024-03-13T21:32:41.635 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.645+0000 7f0b867fc700 1 -- 172.21.15.23:0/1218724619 <== mon.1 v2:172.21.15.23:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+7622 (secure 0 0 0) 0x7f0b880331c0 con 0x7f0b98099a00 2024-03-13T21:32:41.635 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:41.636 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":13,"fsid":"19813cba-5ddc-4ec3-85b1-5e37baeea636","created":"2024-03-13T21:32:30.539980+0000","modified":"2024-03-13T21:32:41.347401+0000","last_up_change":"2024-03-13T21:32:36.292795+0000","last_in_change":"2024-03-13T21:32:36.292795+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-03-13T21:32:33.319887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-13T21:32:39.554513+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a63e0271-0bb4-4f3b-b8a0-681e5b8ce56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6804","nonce":39974}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6805","nonce":39974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6807","nonce":39974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6806","nonce":39974}]},"public_addr":"172.21.15.23:6804/39974","cluster_addr":"172.21.15.23:6805/39974","heartbeat_back_addr":"172.21.15.23:6807/39974","heartbeat_front_addr":"172.21.15.23:6806/39974","state":["exists","up"]},{"osd":1,"uuid":"a721ee5e-d9d6-420e-ba1f-78ff60306e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6800","nonce":39975}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6801","nonce":39975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6803","nonce":39975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6802","nonce":39975}]},"public_addr":"172.21.15.23:6800/39975","cluster_addr":"172.21.15.23:6801/39975","heartbeat_back_addr":"172.21.15.23:6803/39975","heartbeat_front_addr":"172.21.15.23:6802/39975","state":["exists","up"]},{"osd":2,"uuid":"9ce90734-afdb-498c-952c-64a477a8f27a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6808","nonce":39977}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6809","nonce":39977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6811","nonce":39977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6810","nonce":39977}]},"public_addr":"172.21.15.23:6808/39977","cluster_addr":"172.21.15.23:6809/39977","heartbeat_back_addr":"172.21.15.23:6811/39977","heartbeat_front_addr":"172.21.15.23:6810/39977","state":["exists","up"]},{"osd":3,"uuid":"fbfaabf4-47ea-40d2-8c62-3239f195bfe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6812","nonce":39976}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6813","nonce":39976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6815","nonce":39976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6814","nonce":39976}]},"public_addr":"172.21.15.23:6812/39976","cluster_addr":"172.21.15.23:6813/39976","heartbeat_back_addr":"172.21.15.23:6815/39976","heartbeat_front_addr":"172.21.15.23:6814/39976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.474369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.254337+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.761561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.481537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-03-13T21:32:41.637 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.646+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/1218724619 >> v1:172.21.15.23:6816/39358 conn(0x7f0b78038e30 legacy=0x7f0b7803b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.637 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.646+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0b98099a00 msgr2=0x7f0b98095510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.637 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.646+0000 7f0b9f6cc700 1 --2- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0b98099a00 0x7f0b98095510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0b8800c000 tx=0x7f0b88006df0).stop 2024-03-13T21:32:41.637 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.646+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/1218724619 shutdown_connections 2024-03-13T21:32:41.637 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.646+0000 7f0b9f6cc700 1 --2- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0b981366f0 0x7f0b981491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.637 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.646+0000 7f0b9f6cc700 1 --2- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0b98099a00 0x7f0b98095510 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.637 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.646+0000 7f0b9f6cc700 1 --2- 172.21.15.23:0/1218724619 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0b9809b520 0x7f0b98148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.638 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.646+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/1218724619 >> 172.21.15.23:0/1218724619 conn(0x7f0b9806c0c0 msgr2=0x7f0b98133310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:41.638 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.646+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/1218724619 shutdown_connections 2024-03-13T21:32:41.638 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.646+0000 7f0b9f6cc700 1 -- 172.21.15.23:0/1218724619 wait complete. 2024-03-13T21:32:41.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-03-13T21:32:33.319887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-03-13T21:32:39.554513+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-03-13T21:32:41.648 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-03-13T21:32:41.757 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.766+0000 7f075c3f8700 1 Processor -- start 2024-03-13T21:32:41.757 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.766+0000 7f075c3f8700 1 -- start start 2024-03-13T21:32:41.759 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075c3f8700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754133bc0 0x7f0754067570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:41.759 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075c3f8700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f0754139fa0 0x7f075413a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:41.759 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075c3f8700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07540a4c50 con 0x7f07541337e0 2024-03-13T21:32:41.759 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075c3f8700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0754133fe0 con 0x7f0754139fa0 2024-03-13T21:32:41.759 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075c3f8700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07540a4f10 con 0x7f0754133bc0 2024-03-13T21:32:41.759 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f0759993700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754133bc0 0x7f0754067570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075a194700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f07541337e0 0x7f0754139890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59118/0 (socket says 172.21.15.23:59118) 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075a194700 1 -- 172.21.15.23:0/2676731631 learned_addr learned my addr 172.21.15.23:0/2676731631 (peer_addr_for_me v1:172.21.15.23:0/0) 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075a995700 1 --2- 172.21.15.23:0/2676731631 >> v2:172.21.15.23:3300/0 conn(0x7f0754139fa0 0x7f075413a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075a995700 1 -- 172.21.15.23:0/2676731631 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754133bc0 msgr2=0x7f0754067570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075a995700 1 --2- 172.21.15.23:0/2676731631 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754133bc0 0x7f0754067570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075a995700 1 -- 172.21.15.23:0/2676731631 >> v1:172.21.15.23:6789/0 conn(0x7f07541337e0 legacy=0x7f0754139890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f0759192700 1 -- 172.21.15.23:0/2676731631 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275184600 0 0) 0x7f07540a4c50 con 0x7f07541337e0 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075a995700 1 -- 172.21.15.23:0/2676731631 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0754060520 con 0x7f0754139fa0 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f0759993700 1 --2- 172.21.15.23:0/2676731631 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754133bc0 0x7f0754067570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f075a995700 1 --2- 172.21.15.23:0/2676731631 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0754139fa0 0x7f075413a400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f074c0032f0 tx=0x7f074c0091d0).ready entity=mon.0 client_cookie=df3ff2cec1bf9253 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f0759192700 1 -- 172.21.15.23:0/2676731631 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074c00b6c0 con 0x7f0754139fa0 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f0759192700 1 -- 172.21.15.23:0/2676731631 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c00b820 con 0x7f0754139fa0 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.767+0000 7f0759192700 1 -- 172.21.15.23:0/2676731631 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074c00b990 con 0x7f0754139fa0 2024-03-13T21:32:41.760 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.768+0000 7f075c3f8700 1 -- 172.21.15.23:0/2676731631 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0754139fa0 msgr2=0x7f075413a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.761 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.768+0000 7f075c3f8700 1 --2- 172.21.15.23:0/2676731631 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0754139fa0 0x7f075413a400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f074c0032f0 tx=0x7f074c0091d0).stop 2024-03-13T21:32:41.761 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.768+0000 7f075c3f8700 1 -- 172.21.15.23:0/2676731631 shutdown_connections 2024-03-13T21:32:41.761 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.768+0000 7f075c3f8700 1 --2- 172.21.15.23:0/2676731631 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0754139fa0 0x7f075413a400 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.761 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.768+0000 7f075c3f8700 1 --2- 172.21.15.23:0/2676731631 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754133bc0 0x7f0754067570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.761 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.768+0000 7f075c3f8700 1 -- 172.21.15.23:0/2676731631 >> 172.21.15.23:0/2676731631 conn(0x7f07540a93f0 msgr2=0x7f07540a53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:41.761 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.768+0000 7f075c3f8700 1 -- 172.21.15.23:0/2676731631 shutdown_connections 2024-03-13T21:32:41.761 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.768+0000 7f075c3f8700 1 -- 172.21.15.23:0/2676731631 wait complete. 2024-03-13T21:32:41.763 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.769+0000 7f075c3f8700 1 Processor -- start 2024-03-13T21:32:41.763 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.769+0000 7f075c3f8700 1 -- start start 2024-03-13T21:32:41.763 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075c3f8700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f07541337e0 0x7f0754149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:41.763 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075c3f8700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0754133bc0 0x7f0754149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:41.763 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075c3f8700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754139fa0 0x7f0754149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075c3f8700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0754149f60 con 0x7f07541337e0 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075c3f8700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075409fd70 con 0x7f0754139fa0 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075c3f8700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07540a0050 con 0x7f0754133bc0 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f0759993700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0754133bc0 0x7f0754149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075a995700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754139fa0 0x7f0754149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f0759993700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0754133bc0 0x7f0754149580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36338/0 (socket says 172.21.15.23:36338) 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075a194700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f07541337e0 0x7f0754149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f0759993700 1 -- 172.21.15.23:0/2740415626 learned_addr learned my addr 172.21.15.23:0/2740415626 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075a995700 1 --2- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754139fa0 0x7f0754149a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44748/0 (socket says 172.21.15.23:44748) 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f0759993700 1 -- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754139fa0 msgr2=0x7f0754149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f0759993700 1 --2- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754139fa0 0x7f0754149a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f0759993700 1 -- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f07541337e0 msgr2=0x7f0754149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.764 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f0759993700 1 --2- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f07541337e0 0x7f0754149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f0759993700 1 -- 172.21.15.23:0/2740415626 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074400b180 con 0x7f0754133bc0 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075a194700 1 --2- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f07541337e0 0x7f0754149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f075a995700 1 --2- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754139fa0 0x7f0754149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.770+0000 7f0759993700 1 --2- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0754133bc0 0x7f0754149580 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f074400c000 tx=0x7f0744006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.771+0000 7f07437fe700 1 -- 172.21.15.23:0/2740415626 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074400c5f0 con 0x7f0754133bc0 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.771+0000 7f07437fe700 1 -- 172.21.15.23:0/2740415626 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074400c750 con 0x7f0754133bc0 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.771+0000 7f075c3f8700 1 -- 172.21.15.23:0/2740415626 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c009470 con 0x7f0754133bc0 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.771+0000 7f07437fe700 1 -- 172.21.15.23:0/2740415626 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f074400cab0 con 0x7f0754133bc0 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.771+0000 7f075c3f8700 1 -- 172.21.15.23:0/2740415626 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0754135b90 con 0x7f0754133bc0 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.771+0000 7f07437fe700 1 -- 172.21.15.23:0/2740415626 <== mon.2 v2:172.21.15.23:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f074400cc10 con 0x7f0754133bc0 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.772+0000 7f07437fe700 1 -- 172.21.15.23:0/2740415626 <== mon.2 v2:172.21.15.23:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f074404aa40 con 0x7f0754133bc0 2024-03-13T21:32:41.765 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.772+0000 7f075c3f8700 1 -- 172.21.15.23:0/2740415626 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0754065a50 con 0x7f0754133bc0 2024-03-13T21:32:41.766 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.775+0000 7f07437fe700 1 -- 172.21.15.23:0/2740415626 <== mon.2 v2:172.21.15.23:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f07440336c0 con 0x7f0754133bc0 2024-03-13T21:32:41.882 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.891+0000 7f075c3f8700 1 -- 172.21.15.23:0/2740415626 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0754057a60 con 0x7f0754133bc0 2024-03-13T21:32:41.883 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.892+0000 7f07437fe700 1 -- 172.21.15.23:0/2740415626 <== mon.2 v2:172.21.15.23:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v13) v1 ==== 110+0+10 (secure 0 0 0) 0x7f07440332b0 con 0x7f0754133bc0 2024-03-13T21:32:41.883 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 1 2024-03-13T21:32:41.884 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.893+0000 7f075c3f8700 1 -- 172.21.15.23:0/2740415626 >> v1:172.21.15.23:6816/39358 conn(0x7f0734038e30 legacy=0x7f073403b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.884 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.893+0000 7f075c3f8700 1 -- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0754133bc0 msgr2=0x7f0754149580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:41.884 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.893+0000 7f075c3f8700 1 --2- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0754133bc0 0x7f0754149580 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f074400c000 tx=0x7f0744006df0).stop 2024-03-13T21:32:41.885 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.893+0000 7f075c3f8700 1 -- 172.21.15.23:0/2740415626 shutdown_connections 2024-03-13T21:32:41.885 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.893+0000 7f075c3f8700 1 --2- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0754139fa0 0x7f0754149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.885 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.893+0000 7f075c3f8700 1 --2- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f07541337e0 0x7f0754149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.885 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.893+0000 7f075c3f8700 1 --2- 172.21.15.23:0/2740415626 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0754133bc0 0x7f0754149580 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:41.885 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.893+0000 7f075c3f8700 1 -- 172.21.15.23:0/2740415626 >> 172.21.15.23:0/2740415626 conn(0x7f07540a93f0 msgr2=0x7f0754139500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:41.885 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.893+0000 7f075c3f8700 1 -- 172.21.15.23:0/2740415626 shutdown_connections 2024-03-13T21:32:41.885 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:41.893+0000 7f075c3f8700 1 -- 172.21.15.23:0/2740415626 wait complete. 2024-03-13T21:32:41.894 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-03-13T21:32:42.003 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.012+0000 7f0ef2a44700 1 Processor -- start 2024-03-13T21:32:42.003 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.012+0000 7f0ef2a44700 1 -- start start 2024-03-13T21:32:42.006 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef2a44700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec09b520 0x7f0eec09bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:42.006 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef2a44700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f0eec136ad0 0x7f0eec099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:42.006 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef2a44700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eec136f20 con 0x7f0eec1366f0 2024-03-13T21:32:42.006 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef2a44700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eec09a380 con 0x7f0eec136ad0 2024-03-13T21:32:42.007 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef2a44700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec135fe0 con 0x7f0eec09b520 2024-03-13T21:32:42.007 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef0fe1700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f0eec136ad0 0x7f0eec099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:42.007 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0eeb7fe700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec09b520 0x7f0eec09bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:42.007 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0eebfff700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f0eec1366f0 0x7f0eec139530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59122/0 (socket says 172.21.15.23:59122) 2024-03-13T21:32:42.007 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0eeb7fe700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec09b520 0x7f0eec09bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36350/0 (socket says 172.21.15.23:36350) 2024-03-13T21:32:42.007 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0eebfff700 1 -- 172.21.15.23:0/1764533495 learned_addr learned my addr 172.21.15.23:0/1764533495 (peer_addr_for_me v1:172.21.15.23:0/0) 2024-03-13T21:32:42.008 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef0fe1700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f0eec136ad0 0x7f0eec099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49866/0 (socket says 172.21.15.23:49866) 2024-03-13T21:32:42.008 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef0fe1700 1 -- 172.21.15.23:0/1764533495 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec09b520 msgr2=0x7f0eec09bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:42.008 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef0fe1700 1 --2- 172.21.15.23:0/1764533495 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec09b520 0x7f0eec09bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.008 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef0fe1700 1 -- 172.21.15.23:0/1764533495 >> v1:172.21.15.23:6789/0 conn(0x7f0eec1366f0 legacy=0x7f0eec139530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.008 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef0fe1700 1 -- 172.21.15.23:0/1764533495 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec09a0c0 con 0x7f0eec136ad0 2024-03-13T21:32:42.008 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0eeb7fe700 1 --2- 172.21.15.23:0/1764533495 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec09b520 0x7f0eec09bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:42.009 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.013+0000 7f0ef0fe1700 1 --2- 172.21.15.23:0/1764533495 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec136ad0 0x7f0eec099a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0eec05c360 tx=0x7f0ee40091d0).ready entity=mon.0 client_cookie=81d2ef517771f11f server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:42.009 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0eeaffd700 1 -- 172.21.15.23:0/1764533495 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ee400b680 con 0x7f0eec136ad0 2024-03-13T21:32:42.009 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0eeaffd700 1 -- 172.21.15.23:0/1764533495 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee400b7e0 con 0x7f0eec136ad0 2024-03-13T21:32:42.009 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0eeaffd700 1 -- 172.21.15.23:0/1764533495 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ee400b950 con 0x7f0eec136ad0 2024-03-13T21:32:42.009 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0ef2a44700 1 -- 172.21.15.23:0/1764533495 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec136ad0 msgr2=0x7f0eec099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.009 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0ef2a44700 1 --2- 172.21.15.23:0/1764533495 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec136ad0 0x7f0eec099a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0eec05c360 tx=0x7f0ee40091d0).stop 2024-03-13T21:32:42.009 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0ef2a44700 1 -- 172.21.15.23:0/1764533495 shutdown_connections 2024-03-13T21:32:42.010 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0ef2a44700 1 --2- 172.21.15.23:0/1764533495 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec136ad0 0x7f0eec099a00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.010 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0ef2a44700 1 --2- 172.21.15.23:0/1764533495 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec09b520 0x7f0eec09bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.010 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0ef2a44700 1 -- 172.21.15.23:0/1764533495 >> 172.21.15.23:0/1764533495 conn(0x7f0eec06c0c0 msgr2=0x7f0eec06c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:42.010 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0ef2a44700 1 -- 172.21.15.23:0/1764533495 shutdown_connections 2024-03-13T21:32:42.010 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.014+0000 7f0ef2a44700 1 -- 172.21.15.23:0/1764533495 wait complete. 2024-03-13T21:32:42.011 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.015+0000 7f0ef2a44700 1 Processor -- start 2024-03-13T21:32:42.011 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.015+0000 7f0ef2a44700 1 -- start start 2024-03-13T21:32:42.015 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0ef2a44700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0eec09b520 0x7f0eec095820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:42.015 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0ef2a44700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec1366f0 0x7f0eec130e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:42.015 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0ef2a44700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec136ad0 0x7f0eec1312f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:42.015 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0ef2a44700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec097370 con 0x7f0eec1366f0 2024-03-13T21:32:42.015 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0ef2a44700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec096150 con 0x7f0eec09b520 2024-03-13T21:32:42.015 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0ef2a44700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec096410 con 0x7f0eec136ad0 2024-03-13T21:32:42.015 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0eeb7fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec1366f0 0x7f0eec130e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:42.016 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0eebfff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0eec09b520 0x7f0eec095820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:42.016 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0eeb7fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec1366f0 0x7f0eec130e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49870/0 (socket says 172.21.15.23:49870) 2024-03-13T21:32:42.016 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0eeb7fe700 1 -- 172.21.15.23:0/3225933646 learned_addr learned my addr 172.21.15.23:0/3225933646 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:42.016 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0ef0fe1700 1 --2- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec136ad0 0x7f0eec1312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:42.016 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0eeb7fe700 1 -- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec136ad0 msgr2=0x7f0eec1312f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.017 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0eeb7fe700 1 --2- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec136ad0 0x7f0eec1312f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.017 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0eeb7fe700 1 -- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0eec09b520 msgr2=0x7f0eec095820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.017 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0eeb7fe700 1 --2- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0eec09b520 0x7f0eec095820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.017 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0eeb7fe700 1 -- 172.21.15.23:0/3225933646 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc00b120 con 0x7f0eec1366f0 2024-03-13T21:32:42.017 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0eeb7fe700 1 --2- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec1366f0 0x7f0eec130e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0edc00c000 tx=0x7f0edc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:42.017 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0ee97fa700 1 -- 172.21.15.23:0/3225933646 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0edc012600 con 0x7f0eec1366f0 2024-03-13T21:32:42.018 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.016+0000 7f0ee97fa700 1 -- 172.21.15.23:0/3225933646 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0edc012760 con 0x7f0eec1366f0 2024-03-13T21:32:42.018 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.017+0000 7f0ef2a44700 1 -- 172.21.15.23:0/3225933646 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee4009470 con 0x7f0eec1366f0 2024-03-13T21:32:42.018 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.017+0000 7f0ee97fa700 1 -- 172.21.15.23:0/3225933646 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0edc012ac0 con 0x7f0eec1366f0 2024-03-13T21:32:42.018 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.017+0000 7f0ef2a44700 1 -- 172.21.15.23:0/3225933646 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec0947b0 con 0x7f0eec1366f0 2024-03-13T21:32:42.019 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.018+0000 7f0ee97fa700 1 -- 172.21.15.23:0/3225933646 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f0edc022070 con 0x7f0eec1366f0 2024-03-13T21:32:42.019 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.018+0000 7f0ef2a44700 1 -- 172.21.15.23:0/3225933646 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eec0980a0 con 0x7f0eec1366f0 2024-03-13T21:32:42.019 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.018+0000 7f0ee97fa700 1 -- 172.21.15.23:0/3225933646 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f0edc04a880 con 0x7f0eec1366f0 2024-03-13T21:32:42.019 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.021+0000 7f0ee97fa700 1 -- 172.21.15.23:0/3225933646 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f0eec0980a0 con 0x7f0eec1366f0 2024-03-13T21:32:42.129 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.138+0000 7f0ef2a44700 1 -- 172.21.15.23:0/3225933646 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0eec0980a0 con 0x7f0eec1366f0 2024-03-13T21:32:42.129 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.138+0000 7f0ee97fa700 1 -- 172.21.15.23:0/3225933646 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v13) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0eec0980a0 con 0x7f0eec1366f0 2024-03-13T21:32:42.129 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 8 2024-03-13T21:32:42.130 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.139+0000 7f0ef2a44700 1 -- 172.21.15.23:0/3225933646 >> v1:172.21.15.23:6816/39358 conn(0x7f0ed0038e30 legacy=0x7f0ed003b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.131 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.139+0000 7f0ef2a44700 1 -- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec1366f0 msgr2=0x7f0eec130e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.131 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.139+0000 7f0ef2a44700 1 --2- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec1366f0 0x7f0eec130e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0edc00c000 tx=0x7f0edc008040).stop 2024-03-13T21:32:42.131 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.139+0000 7f0ef2a44700 1 -- 172.21.15.23:0/3225933646 shutdown_connections 2024-03-13T21:32:42.131 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.139+0000 7f0ef2a44700 1 --2- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f0eec136ad0 0x7f0eec1312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.131 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.139+0000 7f0ef2a44700 1 --2- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f0eec09b520 0x7f0eec095820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.131 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.139+0000 7f0ef2a44700 1 --2- 172.21.15.23:0/3225933646 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f0eec1366f0 0x7f0eec130e00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.131 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.139+0000 7f0ef2a44700 1 -- 172.21.15.23:0/3225933646 >> 172.21.15.23:0/3225933646 conn(0x7f0eec06c0c0 msgr2=0x7f0eec0663e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:42.131 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.139+0000 7f0ef2a44700 1 -- 172.21.15.23:0/3225933646 shutdown_connections 2024-03-13T21:32:42.131 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.139+0000 7f0ef2a44700 1 -- 172.21.15.23:0/3225933646 wait complete. 2024-03-13T21:32:42.141 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-13T21:32:42.246 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.255+0000 7fe859883700 1 Processor -- start 2024-03-13T21:32:42.247 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.255+0000 7fe859883700 1 -- start start 2024-03-13T21:32:42.249 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.256+0000 7fe859883700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe8541366f0 0x7fe85409bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:42.249 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.256+0000 7fe859883700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fe854136ad0 0x7fe854099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:42.249 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.256+0000 7fe859883700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe854136f40 con 0x7fe85409b520 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.256+0000 7fe859883700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85409a380 con 0x7fe854136ad0 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.256+0000 7fe859883700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe854135fe0 con 0x7fe8541366f0 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.256+0000 7fe8537fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fe854136ad0 0x7fe854099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.256+0000 7fe8527fc700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7fe85409b520 0x7fe854139530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59124/0 (socket says 172.21.15.23:59124) 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.256+0000 7fe852ffd700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe8541366f0 0x7fe85409bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.256+0000 7fe8527fc700 1 -- 172.21.15.23:0/3600136970 learned_addr learned my addr 172.21.15.23:0/3600136970 (peer_addr_for_me v1:172.21.15.23:0/0) 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.256+0000 7fe8537fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fe854136ad0 0x7fe854099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49882/0 (socket says 172.21.15.23:49882) 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe8537fe700 1 -- 172.21.15.23:0/3600136970 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe8541366f0 msgr2=0x7fe85409bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe8537fe700 1 --2- 172.21.15.23:0/3600136970 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe8541366f0 0x7fe85409bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe8537fe700 1 -- 172.21.15.23:0/3600136970 >> v1:172.21.15.23:6789/0 conn(0x7fe85409b520 legacy=0x7fe854139530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe8537fe700 1 -- 172.21.15.23:0/3600136970 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85409a0c0 con 0x7fe854136ad0 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe852ffd700 1 --2- 172.21.15.23:0/3600136970 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe8541366f0 0x7fe85409bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:42.250 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe8537fe700 1 --2- 172.21.15.23:0/3600136970 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe854136ad0 0x7fe854099a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe85405c360 tx=0x7fe84c0091d0).ready entity=mon.0 client_cookie=f8d92cf210caa558 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe851ffb700 1 -- 172.21.15.23:0/3600136970 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe84c00b680 con 0x7fe854136ad0 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe851ffb700 1 -- 172.21.15.23:0/3600136970 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c00b7e0 con 0x7fe854136ad0 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe851ffb700 1 -- 172.21.15.23:0/3600136970 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe84c00b950 con 0x7fe854136ad0 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe859883700 1 -- 172.21.15.23:0/3600136970 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe854136ad0 msgr2=0x7fe854099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.257+0000 7fe859883700 1 --2- 172.21.15.23:0/3600136970 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe854136ad0 0x7fe854099a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe85405c360 tx=0x7fe84c0091d0).stop 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.258+0000 7fe859883700 1 -- 172.21.15.23:0/3600136970 shutdown_connections 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.258+0000 7fe859883700 1 --2- 172.21.15.23:0/3600136970 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe8541366f0 0x7fe85409bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.258+0000 7fe859883700 1 --2- 172.21.15.23:0/3600136970 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe854136ad0 0x7fe854099a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.258+0000 7fe859883700 1 -- 172.21.15.23:0/3600136970 >> 172.21.15.23:0/3600136970 conn(0x7fe85406c0c0 msgr2=0x7fe85406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.258+0000 7fe859883700 1 -- 172.21.15.23:0/3600136970 shutdown_connections 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.258+0000 7fe859883700 1 -- 172.21.15.23:0/3600136970 wait complete. 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.258+0000 7fe859883700 1 Processor -- start 2024-03-13T21:32:42.251 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.258+0000 7fe859883700 1 -- start start 2024-03-13T21:32:42.254 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe859883700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe85409b520 0x7fe8540931a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:42.254 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe859883700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fe8541366f0 0x7fe854093950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:42.254 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe859883700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe854136ad0 0x7fe854139810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:42.254 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe859883700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe854092d10 con 0x7fe854136ad0 2024-03-13T21:32:42.254 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe859883700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540944e0 con 0x7fe85409b520 2024-03-13T21:32:42.254 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe859883700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe854149f70 con 0x7fe8541366f0 2024-03-13T21:32:42.254 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe852ffd700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe85409b520 0x7fe8540931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:42.254 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe8537fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe854136ad0 0x7fe854139810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:42.254 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe852ffd700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe85409b520 0x7fe8540931a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44768/0 (socket says 172.21.15.23:44768) 2024-03-13T21:32:42.254 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe852ffd700 1 -- 172.21.15.23:0/4072240405 learned_addr learned my addr 172.21.15.23:0/4072240405 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe8527fc700 1 --2- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fe8541366f0 0x7fe854093950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe8537fe700 1 -- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fe8541366f0 msgr2=0x7fe854093950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe8537fe700 1 --2- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fe8541366f0 0x7fe854093950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe8537fe700 1 -- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe85409b520 msgr2=0x7fe8540931a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe8537fe700 1 --2- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe85409b520 0x7fe8540931a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe8537fe700 1 -- 172.21.15.23:0/4072240405 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84400b180 con 0x7fe854136ad0 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.259+0000 7fe852ffd700 1 --2- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe85409b520 0x7fe8540931a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.260+0000 7fe8537fe700 1 --2- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe854136ad0 0x7fe854139810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fe85405c360 tx=0x7fe84c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.260+0000 7fe84bfff700 1 -- 172.21.15.23:0/4072240405 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe84c011d30 con 0x7fe854136ad0 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.260+0000 7fe84bfff700 1 -- 172.21.15.23:0/4072240405 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c011e90 con 0x7fe854136ad0 2024-03-13T21:32:42.255 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.260+0000 7fe84bfff700 1 -- 172.21.15.23:0/4072240405 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe84c0205f0 con 0x7fe854136ad0 2024-03-13T21:32:42.256 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.260+0000 7fe859883700 1 -- 172.21.15.23:0/4072240405 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c009470 con 0x7fe854136ad0 2024-03-13T21:32:42.256 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.260+0000 7fe859883700 1 -- 172.21.15.23:0/4072240405 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe854149670 con 0x7fe854136ad0 2024-03-13T21:32:42.256 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.261+0000 7fe84bfff700 1 -- 172.21.15.23:0/4072240405 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fe84c01e070 con 0x7fe854136ad0 2024-03-13T21:32:42.256 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.261+0000 7fe84bfff700 1 -- 172.21.15.23:0/4072240405 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fe84c04b0f0 con 0x7fe854136ad0 2024-03-13T21:32:42.256 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.261+0000 7fe859883700 1 -- 172.21.15.23:0/4072240405 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe854092ec0 con 0x7fe854136ad0 2024-03-13T21:32:42.256 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.264+0000 7fe84bfff700 1 -- 172.21.15.23:0/4072240405 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fe854092ec0 con 0x7fe854136ad0 2024-03-13T21:32:42.406 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.415+0000 7fe859883700 1 -- 172.21.15.23:0/4072240405 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe854092ec0 con 0x7fe854136ad0 2024-03-13T21:32:42.406 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.416+0000 7fe84bfff700 1 -- 172.21.15.23:0/4072240405 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+794 (secure 0 0 0) 0x7fe854092ec0 con 0x7fe854136ad0 2024-03-13T21:32:42.407 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:42.407 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6817/3289485691","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6817","nonce":3289485691}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[]} 2024-03-13T21:32:42.408 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.417+0000 7fe859883700 1 -- 172.21.15.23:0/4072240405 >> v1:172.21.15.23:6816/39358 conn(0x7fe838038da0 legacy=0x7fe83803b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.408 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.417+0000 7fe859883700 1 -- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe854136ad0 msgr2=0x7fe854139810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:42.408 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.417+0000 7fe859883700 1 --2- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe854136ad0 0x7fe854139810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fe85405c360 tx=0x7fe84c016000).stop 2024-03-13T21:32:42.408 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.417+0000 7fe859883700 1 -- 172.21.15.23:0/4072240405 shutdown_connections 2024-03-13T21:32:42.408 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.417+0000 7fe859883700 1 --2- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fe854136ad0 0x7fe854139810 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.408 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.417+0000 7fe859883700 1 --2- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fe85409b520 0x7fe8540931a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.408 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.417+0000 7fe859883700 1 --2- 172.21.15.23:0/4072240405 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fe8541366f0 0x7fe854093950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:42.409 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.417+0000 7fe859883700 1 -- 172.21.15.23:0/4072240405 >> 172.21.15.23:0/4072240405 conn(0x7fe85406c0c0 msgr2=0x7fe8540a72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:42.409 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.417+0000 7fe859883700 1 -- 172.21.15.23:0/4072240405 shutdown_connections 2024-03-13T21:32:42.409 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:42.417+0000 7fe859883700 1 -- 172.21.15.23:0/4072240405 wait complete. 2024-03-13T21:32:42.409 INFO:teuthology.orchestra.run.smithi023.stderr:dumped fsmap epoch 2 2024-03-13T21:32:44.419 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-13T21:32:44.539 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.548+0000 7ff00552a700 1 Processor -- start 2024-03-13T21:32:44.540 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.548+0000 7ff00552a700 1 -- start start 2024-03-13T21:32:44.542 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7ff00552a700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136bf0 0x7ff00005bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:44.542 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7ff00552a700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff000136210 0x7ff00005ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:44.542 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7ff00552a700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7ff00005d370 0x7ff0001397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:44.542 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7ff00552a700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00005b970 con 0x7ff00005d370 2024-03-13T21:32:44.542 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7ff00552a700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff000136710 con 0x7ff000136bf0 2024-03-13T21:32:44.543 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7ff00552a700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001369d0 con 0x7ff000136210 2024-03-13T21:32:44.543 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7feffe7fc700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff000136210 0x7ff00005ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:44.543 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7feffeffd700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136bf0 0x7ff00005bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:44.543 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7fefff7fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7ff00005d370 0x7ff0001397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:44.543 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7feffe7fc700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff000136210 0x7ff00005ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36376/0 (socket says 172.21.15.23:36376) 2024-03-13T21:32:44.543 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7feffeffd700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136bf0 0x7ff00005bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44780/0 (socket says 172.21.15.23:44780) 2024-03-13T21:32:44.544 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.549+0000 7fefff7fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7ff00005d370 0x7ff0001397e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49898/0 (socket says 172.21.15.23:49898) 2024-03-13T21:32:44.544 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffeffd700 1 -- 172.21.15.23:0/138626204 learned_addr learned my addr 172.21.15.23:0/138626204 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:44.544 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffe7fc700 1 -- 172.21.15.23:0/138626204 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136bf0 msgr2=0x7ff00005bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:44.544 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffe7fc700 1 --2- 172.21.15.23:0/138626204 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136bf0 0x7ff00005bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:44.544 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffe7fc700 1 -- 172.21.15.23:0/138626204 >> v2:172.21.15.23:3300/0 conn(0x7ff00005d370 msgr2=0x7ff0001397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:44.544 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffe7fc700 1 --2- 172.21.15.23:0/138626204 >> v2:172.21.15.23:3300/0 conn(0x7ff00005d370 0x7ff0001397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:44.545 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffe7fc700 1 -- 172.21.15.23:0/138626204 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff000136fd0 con 0x7ff000136210 2024-03-13T21:32:44.545 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffeffd700 1 --2- 172.21.15.23:0/138626204 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136bf0 0x7ff00005bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:44.545 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7fefff7fe700 1 --2- 172.21.15.23:0/138626204 >> v2:172.21.15.23:3300/0 conn(0x7ff00005d370 0x7ff0001397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:44.545 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffe7fc700 1 --2- 172.21.15.23:0/138626204 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff000136210 0x7ff00005ce80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7feff0002f70 tx=0x7feff0006df0).ready entity=mon.2 client_cookie=1fca75834aa8a4c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:44.545 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffdffb700 1 -- 172.21.15.23:0/138626204 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff000d4a0 con 0x7ff000136210 2024-03-13T21:32:44.545 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffdffb700 1 -- 172.21.15.23:0/138626204 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff0000e90 con 0x7ff000136210 2024-03-13T21:32:44.546 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.550+0000 7feffdffb700 1 -- 172.21.15.23:0/138626204 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff000d7f0 con 0x7ff000136210 2024-03-13T21:32:44.546 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.551+0000 7ff00552a700 1 -- 172.21.15.23:0/138626204 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff000136210 msgr2=0x7ff00005ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:44.546 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.551+0000 7ff00552a700 1 --2- 172.21.15.23:0/138626204 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff000136210 0x7ff00005ce80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7feff0002f70 tx=0x7feff0006df0).stop 2024-03-13T21:32:44.546 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.551+0000 7ff00552a700 1 -- 172.21.15.23:0/138626204 shutdown_connections 2024-03-13T21:32:44.547 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.551+0000 7ff00552a700 1 --2- 172.21.15.23:0/138626204 >> v2:172.21.15.23:3300/0 conn(0x7ff00005d370 0x7ff0001397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:44.547 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.551+0000 7ff00552a700 1 --2- 172.21.15.23:0/138626204 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136bf0 0x7ff00005bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:44.547 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.551+0000 7ff00552a700 1 --2- 172.21.15.23:0/138626204 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff000136210 0x7ff00005ce80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:44.548 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.551+0000 7ff00552a700 1 -- 172.21.15.23:0/138626204 >> 172.21.15.23:0/138626204 conn(0x7ff000066a50 msgr2=0x7ff00006c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:44.548 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.551+0000 7ff00552a700 1 -- 172.21.15.23:0/138626204 shutdown_connections 2024-03-13T21:32:44.548 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.551+0000 7ff00552a700 1 -- 172.21.15.23:0/138626204 wait complete. 2024-03-13T21:32:44.555 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7ff00552a700 1 Processor -- start 2024-03-13T21:32:44.555 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7ff00552a700 1 -- start start 2024-03-13T21:32:44.555 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7ff00552a700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff00005d370 0x7ff0000a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:44.555 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7ff00552a700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136210 0x7ff0001494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:44.555 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7ff00552a700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff000136bf0 0x7ff0001499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:44.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7ff00552a700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000a42e0 con 0x7ff000136bf0 2024-03-13T21:32:44.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7ff00552a700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff000145ee0 con 0x7ff000136210 2024-03-13T21:32:44.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7ff00552a700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000a5270 con 0x7ff00005d370 2024-03-13T21:32:44.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7feffe7fc700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136210 0x7ff0001494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:44.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7fefff7fe700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff000136bf0 0x7ff0001499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:44.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7feffeffd700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff00005d370 0x7ff0000a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:44.556 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7feffe7fc700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136210 0x7ff0001494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44794/0 (socket says 172.21.15.23:44794) 2024-03-13T21:32:44.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.552+0000 7feffe7fc700 1 -- 172.21.15.23:0/57732443 learned_addr learned my addr 172.21.15.23:0/57732443 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:44.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7feffeffd700 1 -- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136210 msgr2=0x7ff0001494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:44.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7feffeffd700 1 --2- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136210 0x7ff0001494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:44.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7feffeffd700 1 -- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff000136bf0 msgr2=0x7ff0001499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:44.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7feffeffd700 1 --2- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff000136bf0 0x7ff0001499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:44.557 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7feffeffd700 1 -- 172.21.15.23:0/57732443 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff8009470 con 0x7ff00005d370 2024-03-13T21:32:44.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7fefff7fe700 1 --2- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff000136bf0 0x7ff0001499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:44.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7feffeffd700 1 --2- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff00005d370 0x7ff0000a38a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7feff400c000 tx=0x7feff4006eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:44.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7fefeffff700 1 -- 172.21.15.23:0/57732443 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff4012600 con 0x7ff00005d370 2024-03-13T21:32:44.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7fefeffff700 1 -- 172.21.15.23:0/57732443 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff4012760 con 0x7ff00005d370 2024-03-13T21:32:44.558 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7ff00552a700 1 -- 172.21.15.23:0/57732443 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff400b1e0 con 0x7ff00005d370 2024-03-13T21:32:44.559 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7fefeffff700 1 -- 172.21.15.23:0/57732443 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff4012ac0 con 0x7ff00005d370 2024-03-13T21:32:44.559 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.553+0000 7ff00552a700 1 -- 172.21.15.23:0/57732443 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff000b180 con 0x7ff00005d370 2024-03-13T21:32:44.559 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.554+0000 7fefeffff700 1 -- 172.21.15.23:0/57732443 <== mon.2 v2:172.21.15.23:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7feff4012c20 con 0x7ff00005d370 2024-03-13T21:32:44.559 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.554+0000 7fefeffff700 1 -- 172.21.15.23:0/57732443 <== mon.2 v2:172.21.15.23:3302/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7feff404a850 con 0x7ff00005d370 2024-03-13T21:32:44.559 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.554+0000 7ff00552a700 1 -- 172.21.15.23:0/57732443 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff000060150 con 0x7ff00005d370 2024-03-13T21:32:44.559 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.557+0000 7fefeffff700 1 -- 172.21.15.23:0/57732443 <== mon.2 v2:172.21.15.23:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7feff40334d0 con 0x7ff00005d370 2024-03-13T21:32:44.696 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.704+0000 7ff00552a700 1 -- 172.21.15.23:0/57732443 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff000057a60 con 0x7ff00005d370 2024-03-13T21:32:44.696 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.705+0000 7fefeffff700 1 -- 172.21.15.23:0/57732443 <== mon.2 v2:172.21.15.23:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+1086 (secure 0 0 0) 0x7feff4029090 con 0x7ff00005d370 2024-03-13T21:32:44.696 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:44.697 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6817/3289485691","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6817","nonce":3289485691}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6819/1919322381","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6819","nonce":1919322381}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[]} 2024-03-13T21:32:44.699 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.706+0000 7ff00552a700 1 -- 172.21.15.23:0/57732443 >> v1:172.21.15.23:6816/39358 conn(0x7fefdc038da0 legacy=0x7fefdc03b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:44.699 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.706+0000 7ff00552a700 1 -- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff00005d370 msgr2=0x7ff0000a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:44.699 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.706+0000 7ff00552a700 1 --2- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff00005d370 0x7ff0000a38a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7feff400c000 tx=0x7feff4006eb0).stop 2024-03-13T21:32:44.699 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.706+0000 7ff00552a700 1 -- 172.21.15.23:0/57732443 shutdown_connections 2024-03-13T21:32:44.699 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.706+0000 7ff00552a700 1 --2- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7ff000136bf0 0x7ff0001499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:44.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.706+0000 7ff00552a700 1 --2- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7ff00005d370 0x7ff0000a38a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:44.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.706+0000 7ff00552a700 1 --2- 172.21.15.23:0/57732443 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7ff000136210 0x7ff0001494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:44.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.706+0000 7ff00552a700 1 -- 172.21.15.23:0/57732443 >> 172.21.15.23:0/57732443 conn(0x7ff000066a50 msgr2=0x7ff0000a8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:44.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.706+0000 7ff00552a700 1 -- 172.21.15.23:0/57732443 shutdown_connections 2024-03-13T21:32:44.700 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:44.706+0000 7ff00552a700 1 -- 172.21.15.23:0/57732443 wait complete. 2024-03-13T21:32:44.700 INFO:teuthology.orchestra.run.smithi023.stderr:dumped fsmap epoch 3 2024-03-13T21:32:46.708 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-13T21:32:46.827 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.836+0000 7fcf89404700 1 Processor -- start 2024-03-13T21:32:46.827 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.836+0000 7fcf89404700 1 -- start start 2024-03-13T21:32:46.830 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.837+0000 7fcf89404700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fcf84136bf0 0x7fcf8405bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:46.830 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.837+0000 7fcf89404700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf84136210 0x7fcf8405ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:46.830 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.837+0000 7fcf89404700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8405b970 con 0x7fcf8405d370 2024-03-13T21:32:46.830 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.837+0000 7fcf89404700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf84136710 con 0x7fcf84136bf0 2024-03-13T21:32:46.830 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.837+0000 7fcf89404700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf841369d0 con 0x7fcf84136210 2024-03-13T21:32:46.830 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.837+0000 7fcf82ffd700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fcf84136bf0 0x7fcf8405bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:46.830 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.837+0000 7fcf827fc700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf84136210 0x7fcf8405ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:46.830 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.837+0000 7fcf837fe700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7fcf8405d370 0x7fcf841397e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59132/0 (socket says 172.21.15.23:59132) 2024-03-13T21:32:46.830 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.837+0000 7fcf837fe700 1 -- 172.21.15.23:0/2896023387 learned_addr learned my addr 172.21.15.23:0/2896023387 (peer_addr_for_me v1:172.21.15.23:0/0) 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.838+0000 7fcf82ffd700 1 -- 172.21.15.23:0/2896023387 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf84136210 msgr2=0x7fcf8405ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.838+0000 7fcf82ffd700 1 --2- 172.21.15.23:0/2896023387 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf84136210 0x7fcf8405ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.838+0000 7fcf82ffd700 1 -- 172.21.15.23:0/2896023387 >> v1:172.21.15.23:6789/0 conn(0x7fcf8405d370 legacy=0x7fcf841397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.838+0000 7fcf82ffd700 1 -- 172.21.15.23:0/2896023387 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf84136fd0 con 0x7fcf84136bf0 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.838+0000 7fcf81ffb700 1 -- 172.21.15.23:0/2896023387 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631746467 0 0) 0x7fcf8405b970 con 0x7fcf8405d370 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.838+0000 7fcf827fc700 1 --2- 172.21.15.23:0/2896023387 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf84136210 0x7fcf8405ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.838+0000 7fcf82ffd700 1 --2- 172.21.15.23:0/2896023387 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136bf0 0x7fcf8405bde0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fcf78002f70 tx=0x7fcf78006df0).ready entity=mon.0 client_cookie=32c7e33a770f55cc server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.838+0000 7fcf81ffb700 1 -- 172.21.15.23:0/2896023387 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf7800d570 con 0x7fcf84136bf0 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.838+0000 7fcf81ffb700 1 -- 172.21.15.23:0/2896023387 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf78000e90 con 0x7fcf84136bf0 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.838+0000 7fcf81ffb700 1 -- 172.21.15.23:0/2896023387 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf7800d8c0 con 0x7fcf84136bf0 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.839+0000 7fcf89404700 1 -- 172.21.15.23:0/2896023387 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136bf0 msgr2=0x7fcf8405bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.839+0000 7fcf89404700 1 --2- 172.21.15.23:0/2896023387 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136bf0 0x7fcf8405bde0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fcf78002f70 tx=0x7fcf78006df0).stop 2024-03-13T21:32:46.831 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.839+0000 7fcf89404700 1 -- 172.21.15.23:0/2896023387 shutdown_connections 2024-03-13T21:32:46.832 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.839+0000 7fcf89404700 1 --2- 172.21.15.23:0/2896023387 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136bf0 0x7fcf8405bde0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:46.832 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.839+0000 7fcf89404700 1 --2- 172.21.15.23:0/2896023387 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf84136210 0x7fcf8405ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:46.832 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.839+0000 7fcf89404700 1 -- 172.21.15.23:0/2896023387 >> 172.21.15.23:0/2896023387 conn(0x7fcf84066a50 msgr2=0x7fcf8406c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:46.832 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.839+0000 7fcf89404700 1 -- 172.21.15.23:0/2896023387 shutdown_connections 2024-03-13T21:32:46.832 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.839+0000 7fcf89404700 1 -- 172.21.15.23:0/2896023387 wait complete. 2024-03-13T21:32:46.834 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf89404700 1 Processor -- start 2024-03-13T21:32:46.834 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf89404700 1 -- start start 2024-03-13T21:32:46.834 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf89404700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf8405d370 0x7fcf840a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:46.834 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf89404700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136210 0x7fcf840a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:46.834 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf89404700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fcf84136bf0 0x7fcf84145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:46.834 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf89404700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840a1e20 con 0x7fcf84136210 2024-03-13T21:32:46.834 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf89404700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84145d40 con 0x7fcf8405d370 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf89404700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840a0aa0 con 0x7fcf84136bf0 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf82ffd700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf8405d370 0x7fcf840a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf827fc700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136210 0x7fcf840a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf82ffd700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf8405d370 0x7fcf840a2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44814/0 (socket says 172.21.15.23:44814) 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf827fc700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136210 0x7fcf840a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49922/0 (socket says 172.21.15.23:49922) 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf837fe700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fcf84136bf0 0x7fcf84145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.840+0000 7fcf82ffd700 1 -- 172.21.15.23:0/229411090 learned_addr learned my addr 172.21.15.23:0/229411090 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf82ffd700 1 -- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fcf84136bf0 msgr2=0x7fcf84145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf82ffd700 1 --2- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fcf84136bf0 0x7fcf84145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf82ffd700 1 -- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136210 msgr2=0x7fcf840a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf82ffd700 1 --2- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136210 0x7fcf840a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf82ffd700 1 -- 172.21.15.23:0/229411090 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7400b180 con 0x7fcf8405d370 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf827fc700 1 --2- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136210 0x7fcf840a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:46.835 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf82ffd700 1 --2- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf8405d370 0x7fcf840a2730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fcf840a24a0 tx=0x7fcf78014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:46.836 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf73fff700 1 -- 172.21.15.23:0/229411090 <== mon.1 v2:172.21.15.23:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf78016070 con 0x7fcf8405d370 2024-03-13T21:32:46.836 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf73fff700 1 -- 172.21.15.23:0/229411090 <== mon.1 v2:172.21.15.23:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf780138d0 con 0x7fcf8405d370 2024-03-13T21:32:46.836 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf89404700 1 -- 172.21.15.23:0/229411090 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7800b180 con 0x7fcf8405d370 2024-03-13T21:32:46.836 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf73fff700 1 -- 172.21.15.23:0/229411090 <== mon.1 v2:172.21.15.23:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf78013c30 con 0x7fcf8405d370 2024-03-13T21:32:46.836 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.841+0000 7fcf89404700 1 -- 172.21.15.23:0/229411090 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf84139c80 con 0x7fcf8405d370 2024-03-13T21:32:46.836 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.842+0000 7fcf73fff700 1 -- 172.21.15.23:0/229411090 <== mon.1 v2:172.21.15.23:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fcf78025030 con 0x7fcf8405d370 2024-03-13T21:32:46.836 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.842+0000 7fcf73fff700 1 -- 172.21.15.23:0/229411090 <== mon.1 v2:172.21.15.23:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fcf7804b3d0 con 0x7fcf8405d370 2024-03-13T21:32:46.836 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.842+0000 7fcf89404700 1 -- 172.21.15.23:0/229411090 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf840a0dd0 con 0x7fcf8405d370 2024-03-13T21:32:46.836 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.845+0000 7fcf73fff700 1 -- 172.21.15.23:0/229411090 <== mon.1 v2:172.21.15.23:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fcf78023030 con 0x7fcf8405d370 2024-03-13T21:32:46.984 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.993+0000 7fcf89404700 1 -- 172.21.15.23:0/229411090 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf84060150 con 0x7fcf8405d370 2024-03-13T21:32:46.984 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.994+0000 7fcf73fff700 1 -- 172.21.15.23:0/229411090 <== mon.1 v2:172.21.15.23:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+1378 (secure 0 0 0) 0x7fcf78028610 con 0x7fcf8405d370 2024-03-13T21:32:46.985 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:46.985 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6817/3289485691","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6817","nonce":3289485691}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.23:6818/1584645510","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6818","nonce":1584645510}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":4},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.23:6819/1919322381","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6819","nonce":1919322381}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[]} 2024-03-13T21:32:46.986 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.995+0000 7fcf89404700 1 -- 172.21.15.23:0/229411090 >> v1:172.21.15.23:6816/39358 conn(0x7fcf60038e30 legacy=0x7fcf6003b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:46.986 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.995+0000 7fcf89404700 1 -- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf8405d370 msgr2=0x7fcf840a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:46.986 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.995+0000 7fcf89404700 1 --2- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf8405d370 0x7fcf840a2730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fcf840a24a0 tx=0x7fcf78014000).stop 2024-03-13T21:32:46.986 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.995+0000 7fcf89404700 1 -- 172.21.15.23:0/229411090 shutdown_connections 2024-03-13T21:32:46.986 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.995+0000 7fcf89404700 1 --2- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fcf84136bf0 0x7fcf84145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:46.987 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.995+0000 7fcf89404700 1 --2- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fcf8405d370 0x7fcf840a2730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:46.987 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.995+0000 7fcf89404700 1 --2- 172.21.15.23:0/229411090 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fcf84136210 0x7fcf840a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:46.987 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.995+0000 7fcf89404700 1 -- 172.21.15.23:0/229411090 >> 172.21.15.23:0/229411090 conn(0x7fcf84066a50 msgr2=0x7fcf840a8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:46.987 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.995+0000 7fcf89404700 1 -- 172.21.15.23:0/229411090 shutdown_connections 2024-03-13T21:32:46.987 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:46.995+0000 7fcf89404700 1 -- 172.21.15.23:0/229411090 wait complete. 2024-03-13T21:32:46.987 INFO:teuthology.orchestra.run.smithi023.stderr:dumped fsmap epoch 4 2024-03-13T21:32:46.997 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T21:32:47.102 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.111+0000 7f9a52d37700 1 Processor -- start 2024-03-13T21:32:47.102 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.111+0000 7f9a52d37700 1 -- start start 2024-03-13T21:32:47.105 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.112+0000 7f9a52d37700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c09f5d0 0x7f9a4c13b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.105 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.112+0000 7f9a52d37700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 0x7f9a4c13da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.105 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.112+0000 7f9a52d37700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c0a1f10 con 0x7f9a4c05dfb0 2024-03-13T21:32:47.105 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.112+0000 7f9a52d37700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0a0f60 con 0x7f9a4c13b6a0 2024-03-13T21:32:47.105 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.112+0000 7f9a52d37700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0a1200 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.105 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.112+0000 7f9a4bfff700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c09f5d0 0x7f9a4c13b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.105 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.112+0000 7f9a512d4700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 0x7f9a4c13da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.105 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.112+0000 7f9a50ad3700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f9a4c05dfb0 0x7f9a4c0a2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59142/0 (socket says 172.21.15.23:59142) 2024-03-13T21:32:47.105 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.112+0000 7f9a50ad3700 1 -- 172.21.15.23:0/447676069 learned_addr learned my addr 172.21.15.23:0/447676069 (peer_addr_for_me v1:172.21.15.23:0/0) 2024-03-13T21:32:47.105 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a512d4700 1 -- 172.21.15.23:0/447676069 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c09f5d0 msgr2=0x7f9a4c13b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a512d4700 1 --2- 172.21.15.23:0/447676069 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c09f5d0 0x7f9a4c13b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a512d4700 1 -- 172.21.15.23:0/447676069 >> v1:172.21.15.23:6789/0 conn(0x7f9a4c05dfb0 legacy=0x7f9a4c0a2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a4b7fe700 1 -- 172.21.15.23:0/447676069 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217465446 0 0) 0x7f9a4c0a1f10 con 0x7f9a4c05dfb0 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a512d4700 1 -- 172.21.15.23:0/447676069 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c060520 con 0x7f9a4c13b6a0 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a4bfff700 1 --2- 172.21.15.23:0/447676069 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c09f5d0 0x7f9a4c13b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a512d4700 1 --2- 172.21.15.23:0/447676069 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 0x7f9a4c13da90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9a440032f0 tx=0x7f9a440091d0).ready entity=mon.1 client_cookie=f1bd217428cdd1a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a4b7fe700 1 -- 172.21.15.23:0/447676069 <== mon.1 v2:172.21.15.23:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a4400b6c0 con 0x7f9a4c13b6a0 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a4b7fe700 1 -- 172.21.15.23:0/447676069 <== mon.1 v2:172.21.15.23:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4400b820 con 0x7f9a4c13b6a0 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a4b7fe700 1 -- 172.21.15.23:0/447676069 <== mon.1 v2:172.21.15.23:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a4400b990 con 0x7f9a4c13b6a0 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a52d37700 1 -- 172.21.15.23:0/447676069 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 msgr2=0x7f9a4c13da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a52d37700 1 --2- 172.21.15.23:0/447676069 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 0x7f9a4c13da90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9a440032f0 tx=0x7f9a440091d0).stop 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a52d37700 1 -- 172.21.15.23:0/447676069 shutdown_connections 2024-03-13T21:32:47.106 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a52d37700 1 --2- 172.21.15.23:0/447676069 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 0x7f9a4c13da90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.107 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a52d37700 1 --2- 172.21.15.23:0/447676069 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c09f5d0 0x7f9a4c13b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.107 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a52d37700 1 -- 172.21.15.23:0/447676069 >> 172.21.15.23:0/447676069 conn(0x7f9a4c0a93f0 msgr2=0x7f9a4c0a53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:47.107 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a52d37700 1 -- 172.21.15.23:0/447676069 shutdown_connections 2024-03-13T21:32:47.107 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.113+0000 7f9a52d37700 1 -- 172.21.15.23:0/447676069 wait complete. 2024-03-13T21:32:47.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.114+0000 7f9a52d37700 1 Processor -- start 2024-03-13T21:32:47.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.114+0000 7f9a52d37700 1 -- start start 2024-03-13T21:32:47.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.115+0000 7f9a52d37700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c05dfb0 0x7f9a4c0a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.115+0000 7f9a52d37700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a4c09f5d0 0x7f9a4c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.115+0000 7f9a52d37700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 0x7f9a4c0a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.115+0000 7f9a52d37700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0a41c0 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.115+0000 7f9a52d37700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c066ca0 con 0x7f9a4c13b6a0 2024-03-13T21:32:47.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.115+0000 7f9a52d37700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0664c0 con 0x7f9a4c05dfb0 2024-03-13T21:32:47.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.115+0000 7f9a50ad3700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c05dfb0 0x7f9a4c0a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.110 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a50ad3700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c05dfb0 0x7f9a4c0a32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36420/0 (socket says 172.21.15.23:36420) 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a512d4700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 0x7f9a4c0a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a50ad3700 1 -- 172.21.15.23:0/3436286524 learned_addr learned my addr 172.21.15.23:0/3436286524 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a4bfff700 1 --2- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a4c09f5d0 0x7f9a4c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a4bfff700 1 -- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c05dfb0 msgr2=0x7f9a4c0a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a4bfff700 1 --2- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c05dfb0 0x7f9a4c0a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a4bfff700 1 -- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 msgr2=0x7f9a4c0a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a4bfff700 1 --2- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 0x7f9a4c0a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a4bfff700 1 -- 172.21.15.23:0/3436286524 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3800b180 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a50ad3700 1 --2- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c05dfb0 0x7f9a4c0a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a4bfff700 1 --2- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a4c09f5d0 0x7f9a4c0a37e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f9a4c0658e0 tx=0x7f9a38006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a49ffb700 1 -- 172.21.15.23:0/3436286524 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a38010780 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a49ffb700 1 -- 172.21.15.23:0/3436286524 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a380108e0 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.111 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a49ffb700 1 -- 172.21.15.23:0/3436286524 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a38010c40 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.112 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a52d37700 1 -- 172.21.15.23:0/3436286524 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a44009470 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.112 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.116+0000 7f9a52d37700 1 -- 172.21.15.23:0/3436286524 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c0645d0 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.112 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.117+0000 7f9a49ffb700 1 -- 172.21.15.23:0/3436286524 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f9a3801c460 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.112 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.117+0000 7f9a52d37700 1 -- 172.21.15.23:0/3436286524 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c066e70 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.112 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.117+0000 7f9a49ffb700 1 -- 172.21.15.23:0/3436286524 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f9a3804aa80 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.112 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.120+0000 7f9a49ffb700 1 -- 172.21.15.23:0/3436286524 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f9a38033570 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.238+0000 7f9a52d37700 1 -- 172.21.15.23:0/3436286524 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a4c057a60 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.229 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.238+0000 7f9a49ffb700 1 -- 172.21.15.23:0/3436286524 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7609 (secure 0 0 0) 0x7f9a38033160 con 0x7f9a4c09f5d0 2024-03-13T21:32:47.229 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:47.229 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":14,"fsid":"19813cba-5ddc-4ec3-85b1-5e37baeea636","created":"2024-03-13T21:32:30.539980+0000","modified":"2024-03-13T21:32:42.359090+0000","last_up_change":"2024-03-13T21:32:36.292795+0000","last_in_change":"2024-03-13T21:32:36.292795+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-03-13T21:32:33.319887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-13T21:32:39.554513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a63e0271-0bb4-4f3b-b8a0-681e5b8ce56b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6804","nonce":39974}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6805","nonce":39974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6807","nonce":39974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6806","nonce":39974}]},"public_addr":"172.21.15.23:6804/39974","cluster_addr":"172.21.15.23:6805/39974","heartbeat_back_addr":"172.21.15.23:6807/39974","heartbeat_front_addr":"172.21.15.23:6806/39974","state":["exists","up"]},{"osd":1,"uuid":"a721ee5e-d9d6-420e-ba1f-78ff60306e44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6800","nonce":39975}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6801","nonce":39975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6803","nonce":39975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6802","nonce":39975}]},"public_addr":"172.21.15.23:6800/39975","cluster_addr":"172.21.15.23:6801/39975","heartbeat_back_addr":"172.21.15.23:6803/39975","heartbeat_front_addr":"172.21.15.23:6802/39975","state":["exists","up"]},{"osd":2,"uuid":"9ce90734-afdb-498c-952c-64a477a8f27a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6808","nonce":39977}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6809","nonce":39977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6811","nonce":39977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6810","nonce":39977}]},"public_addr":"172.21.15.23:6808/39977","cluster_addr":"172.21.15.23:6809/39977","heartbeat_back_addr":"172.21.15.23:6811/39977","heartbeat_front_addr":"172.21.15.23:6810/39977","state":["exists","up"]},{"osd":3,"uuid":"fbfaabf4-47ea-40d2-8c62-3239f195bfe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6812","nonce":39976}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6813","nonce":39976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6815","nonce":39976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.23:6814","nonce":39976}]},"public_addr":"172.21.15.23:6812/39976","cluster_addr":"172.21.15.23:6813/39976","heartbeat_back_addr":"172.21.15.23:6815/39976","heartbeat_front_addr":"172.21.15.23:6814/39976","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.474369+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.254337+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.761561+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T21:32:34.481537+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2024-03-13T21:32:47.231 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.240+0000 7f9a52d37700 1 -- 172.21.15.23:0/3436286524 >> v1:172.21.15.23:6816/39358 conn(0x7f9a2c038f50 legacy=0x7f9a2c03b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.231 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.240+0000 7f9a52d37700 1 -- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a4c09f5d0 msgr2=0x7f9a4c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.231 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.240+0000 7f9a52d37700 1 --2- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a4c09f5d0 0x7f9a4c0a37e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f9a4c0658e0 tx=0x7f9a38006df0).stop 2024-03-13T21:32:47.231 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.240+0000 7f9a52d37700 1 -- 172.21.15.23:0/3436286524 shutdown_connections 2024-03-13T21:32:47.231 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.240+0000 7f9a52d37700 1 --2- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9a4c13b6a0 0x7f9a4c0a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.232 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.240+0000 7f9a52d37700 1 --2- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9a4c05dfb0 0x7f9a4c0a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.232 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.240+0000 7f9a52d37700 1 --2- 172.21.15.23:0/3436286524 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9a4c09f5d0 0x7f9a4c0a37e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.232 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.240+0000 7f9a52d37700 1 -- 172.21.15.23:0/3436286524 >> 172.21.15.23:0/3436286524 conn(0x7f9a4c0a93f0 msgr2=0x7f9a4c05beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:47.232 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.240+0000 7f9a52d37700 1 -- 172.21.15.23:0/3436286524 shutdown_connections 2024-03-13T21:32:47.232 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.240+0000 7f9a52d37700 1 -- 172.21.15.23:0/3436286524 wait complete. 2024-03-13T21:32:47.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-03-13T21:32:33.319887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-03-13T21:32:39.554513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-03-13T21:32:47.242 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-03-13T21:32:47.353 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.363+0000 7fb6cb9d0700 1 Processor -- start 2024-03-13T21:32:47.354 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.363+0000 7fb6cb9d0700 1 -- start start 2024-03-13T21:32:47.356 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.363+0000 7fb6cb9d0700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fb6c41337e0 0x7fb6c4133bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.356 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.363+0000 7fb6cb9d0700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c4139890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.356 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.363+0000 7fb6cb9d0700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c4139d80 0x7fb6c413a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.356 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.363+0000 7fb6cb9d0700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c4067990 con 0x7fb6c41337e0 2024-03-13T21:32:47.356 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.363+0000 7fb6cb9d0700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c40a4c50 con 0x7fb6c4139d80 2024-03-13T21:32:47.356 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.363+0000 7fb6cb9d0700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c40a4f10 con 0x7fb6c4067570 2024-03-13T21:32:47.356 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c8f6b700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c4139890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.356 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c9f6d700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c4139d80 0x7fb6c413a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c976c700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fb6c41337e0 0x7fb6c4133bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c976c700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fb6c41337e0 0x7fb6c4133bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49944/0 (socket says 172.21.15.23:49944) 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c9f6d700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c4139d80 0x7fb6c413a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44846/0 (socket says 172.21.15.23:44846) 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c976c700 1 -- 172.21.15.23:0/70396247 learned_addr learned my addr 172.21.15.23:0/70396247 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c8f6b700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c4139890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36426/0 (socket says 172.21.15.23:36426) 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c976c700 1 -- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 msgr2=0x7fb6c4139890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c976c700 1 --2- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c4139890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c976c700 1 -- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c4139d80 msgr2=0x7fb6c413a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c976c700 1 --2- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c4139d80 0x7fb6c413a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c976c700 1 -- 172.21.15.23:0/70396247 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c413a6d0 con 0x7fb6c41337e0 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c8f6b700 1 --2- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c4139890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c9f6d700 1 --2- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c4139d80 0x7fb6c413a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:47.357 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6c976c700 1 --2- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c41337e0 0x7fb6c4133bc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fb6c0002f70 tx=0x7fb6c0006eb0).ready entity=mon.0 client_cookie=603c24a31823ab13 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6b3fff700 1 -- 172.21.15.23:0/70396247 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c000d570 con 0x7fb6c41337e0 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6b3fff700 1 -- 172.21.15.23:0/70396247 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c0000e90 con 0x7fb6c41337e0 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.364+0000 7fb6b3fff700 1 -- 172.21.15.23:0/70396247 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c000d900 con 0x7fb6c41337e0 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.365+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c41337e0 msgr2=0x7fb6c4133bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.365+0000 7fb6cb9d0700 1 --2- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c41337e0 0x7fb6c4133bc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fb6c0002f70 tx=0x7fb6c0006eb0).stop 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.365+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/70396247 shutdown_connections 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.365+0000 7fb6cb9d0700 1 --2- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c4139d80 0x7fb6c413a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.365+0000 7fb6cb9d0700 1 --2- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c41337e0 0x7fb6c4133bc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.365+0000 7fb6cb9d0700 1 --2- 172.21.15.23:0/70396247 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c4139890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.365+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/70396247 >> 172.21.15.23:0/70396247 conn(0x7fb6c40a93f0 msgr2=0x7fb6c40a53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.365+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/70396247 shutdown_connections 2024-03-13T21:32:47.358 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.365+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/70396247 wait complete. 2024-03-13T21:32:47.360 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6cb9d0700 1 Processor -- start 2024-03-13T21:32:47.360 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6cb9d0700 1 -- start start 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6cb9d0700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c40a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6cb9d0700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c41337e0 0x7fb6c4132510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6cb9d0700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c4139d80 0x7fb6c4132a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6cb9d0700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c40a0380 con 0x7fb6c4139d80 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6cb9d0700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c40a01f0 con 0x7fb6c41337e0 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6cb9d0700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c4093170 con 0x7fb6c4067570 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6c976c700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c40a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6c9f6d700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c4139d80 0x7fb6c4132a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6c976c700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c40a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36436/0 (socket says 172.21.15.23:36436) 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6c976c700 1 -- 172.21.15.23:0/729506473 learned_addr learned my addr 172.21.15.23:0/729506473 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.366+0000 7fb6c9f6d700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c4139d80 0x7fb6c4132a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49948/0 (socket says 172.21.15.23:49948) 2024-03-13T21:32:47.361 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6c8f6b700 1 --2- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c41337e0 0x7fb6c4132510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6c976c700 1 -- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c41337e0 msgr2=0x7fb6c4132510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6c976c700 1 --2- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c41337e0 0x7fb6c4132510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6c976c700 1 -- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c4139d80 msgr2=0x7fb6c4132a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6c976c700 1 --2- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c4139d80 0x7fb6c4132a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6c976c700 1 -- 172.21.15.23:0/729506473 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b400b180 con 0x7fb6c4067570 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6c8f6b700 1 --2- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c41337e0 0x7fb6c4132510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6c9f6d700 1 --2- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c4139d80 0x7fb6c4132a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6c976c700 1 --2- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c40a1130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fb6c0002f70 tx=0x7fb6c0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6b27fc700 1 -- 172.21.15.23:0/729506473 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c000dea0 con 0x7fb6c4067570 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6b27fc700 1 -- 172.21.15.23:0/729506473 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c0006b40 con 0x7fb6c4067570 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/729506473 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6bc009470 con 0x7fb6c4067570 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6b27fc700 1 -- 172.21.15.23:0/729506473 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6c00068b0 con 0x7fb6c4067570 2024-03-13T21:32:47.362 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.367+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/729506473 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c000b1e0 con 0x7fb6c4067570 2024-03-13T21:32:47.363 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.368+0000 7fb6b27fc700 1 -- 172.21.15.23:0/729506473 <== mon.2 v2:172.21.15.23:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fb6c0025030 con 0x7fb6c4067570 2024-03-13T21:32:47.363 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.368+0000 7fb6b27fc700 1 -- 172.21.15.23:0/729506473 <== mon.2 v2:172.21.15.23:3302/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fb6c004b8a0 con 0x7fb6c4067570 2024-03-13T21:32:47.363 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.368+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/729506473 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c409de00 con 0x7fb6c4067570 2024-03-13T21:32:47.363 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.371+0000 7fb6b27fc700 1 -- 172.21.15.23:0/729506473 <== mon.2 v2:172.21.15.23:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fb6c0023070 con 0x7fb6c4067570 2024-03-13T21:32:47.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.489+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/729506473 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb6c40a0f80 con 0x7fb6c4067570 2024-03-13T21:32:47.480 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.489+0000 7fb6b27fc700 1 -- 172.21.15.23:0/729506473 <== mon.2 v2:172.21.15.23:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v14) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb6c0023020 con 0x7fb6c4067570 2024-03-13T21:32:47.480 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 1 2024-03-13T21:32:47.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.491+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/729506473 >> v1:172.21.15.23:6816/39358 conn(0x7fb6a4038da0 legacy=0x7fb6a403b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.491+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 msgr2=0x7fb6c40a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.491+0000 7fb6cb9d0700 1 --2- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c40a1130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fb6c0002f70 tx=0x7fb6c0015000).stop 2024-03-13T21:32:47.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.491+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/729506473 shutdown_connections 2024-03-13T21:32:47.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.491+0000 7fb6cb9d0700 1 --2- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb6c4139d80 0x7fb6c4132a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.491+0000 7fb6cb9d0700 1 --2- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb6c4067570 0x7fb6c40a1130 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.491+0000 7fb6cb9d0700 1 --2- 172.21.15.23:0/729506473 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb6c41337e0 0x7fb6c4132510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.491+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/729506473 >> 172.21.15.23:0/729506473 conn(0x7fb6c40a93f0 msgr2=0x7fb6c4135c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:47.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.491+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/729506473 shutdown_connections 2024-03-13T21:32:47.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.491+0000 7fb6cb9d0700 1 -- 172.21.15.23:0/729506473 wait complete. 2024-03-13T21:32:47.492 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-03-13T21:32:47.600 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.610+0000 7fb9969d8700 1 Processor -- start 2024-03-13T21:32:47.601 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.610+0000 7fb9969d8700 1 -- start start 2024-03-13T21:32:47.603 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.610+0000 7fb9969d8700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb9901366f0 0x7fb99009bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.603 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.610+0000 7fb9969d8700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fb990136ad0 0x7fb990099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.603 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.610+0000 7fb9969d8700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb990136f40 con 0x7fb99009b520 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.610+0000 7fb9969d8700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99009a380 con 0x7fb990136ad0 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.610+0000 7fb9969d8700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb990135fe0 con 0x7fb9901366f0 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.610+0000 7fb98ffff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb9901366f0 0x7fb99009bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.610+0000 7fb994f75700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fb990136ad0 0x7fb990099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb98ffff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb9901366f0 0x7fb99009bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44856/0 (socket says 172.21.15.23:44856) 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.610+0000 7fb987fff700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7fb99009b520 0x7fb990139530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59144/0 (socket says 172.21.15.23:59144) 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb994f75700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7fb990136ad0 0x7fb990099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49964/0 (socket says 172.21.15.23:49964) 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb98ffff700 1 -- 172.21.15.23:0/4242874142 learned_addr learned my addr 172.21.15.23:0/4242874142 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb994f75700 1 -- 172.21.15.23:0/4242874142 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb9901366f0 msgr2=0x7fb99009bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb994f75700 1 --2- 172.21.15.23:0/4242874142 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb9901366f0 0x7fb99009bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb994f75700 1 -- 172.21.15.23:0/4242874142 >> v1:172.21.15.23:6789/0 conn(0x7fb99009b520 legacy=0x7fb990139530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb994f75700 1 -- 172.21.15.23:0/4242874142 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99009a0c0 con 0x7fb990136ad0 2024-03-13T21:32:47.604 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb98f7fe700 1 -- 172.21.15.23:0/4242874142 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479829362 0 0) 0x7fb990136f40 con 0x7fb99009b520 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb98ffff700 1 --2- 172.21.15.23:0/4242874142 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb9901366f0 0x7fb99009bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb994f75700 1 --2- 172.21.15.23:0/4242874142 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb990136ad0 0x7fb990099a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb99005c360 tx=0x7fb9880091d0).ready entity=mon.0 client_cookie=6fedb374fd8c79e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb98f7fe700 1 -- 172.21.15.23:0/4242874142 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb98800b680 con 0x7fb990136ad0 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb98f7fe700 1 -- 172.21.15.23:0/4242874142 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98800b7e0 con 0x7fb990136ad0 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.611+0000 7fb98f7fe700 1 -- 172.21.15.23:0/4242874142 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb98800b950 con 0x7fb990136ad0 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.612+0000 7fb9969d8700 1 -- 172.21.15.23:0/4242874142 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb990136ad0 msgr2=0x7fb990099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.612+0000 7fb9969d8700 1 --2- 172.21.15.23:0/4242874142 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb990136ad0 0x7fb990099a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb99005c360 tx=0x7fb9880091d0).stop 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.612+0000 7fb9969d8700 1 -- 172.21.15.23:0/4242874142 shutdown_connections 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.612+0000 7fb9969d8700 1 --2- 172.21.15.23:0/4242874142 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb9901366f0 0x7fb99009bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.612+0000 7fb9969d8700 1 --2- 172.21.15.23:0/4242874142 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb990136ad0 0x7fb990099a00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.612+0000 7fb9969d8700 1 -- 172.21.15.23:0/4242874142 >> 172.21.15.23:0/4242874142 conn(0x7fb99006c0c0 msgr2=0x7fb99006c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.612+0000 7fb9969d8700 1 -- 172.21.15.23:0/4242874142 shutdown_connections 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.612+0000 7fb9969d8700 1 -- 172.21.15.23:0/4242874142 wait complete. 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb9969d8700 1 Processor -- start 2024-03-13T21:32:47.605 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb9969d8700 1 -- start start 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb9969d8700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb99009b520 0x7fb990148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb9969d8700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb9901366f0 0x7fb990093660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb9969d8700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb990136ad0 0x7fb990094fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb9969d8700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb990093df0 con 0x7fb990136ad0 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb9969d8700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb990149120 con 0x7fb99009b520 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb9969d8700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb990092d00 con 0x7fb9901366f0 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb98ffff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb99009b520 0x7fb990148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb994f75700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb990136ad0 0x7fb990094fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb987fff700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb9901366f0 0x7fb990093660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb98ffff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb99009b520 0x7fb990148c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44866/0 (socket says 172.21.15.23:44866) 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.613+0000 7fb98ffff700 1 -- 172.21.15.23:0/1738705638 learned_addr learned my addr 172.21.15.23:0/1738705638 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:47.608 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb994f75700 1 -- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb9901366f0 msgr2=0x7fb990093660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb994f75700 1 --2- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb9901366f0 0x7fb990093660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb994f75700 1 -- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb99009b520 msgr2=0x7fb990148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb994f75700 1 --2- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb99009b520 0x7fb990148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb994f75700 1 -- 172.21.15.23:0/1738705638 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98000b1e0 con 0x7fb990136ad0 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb98ffff700 1 --2- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb99009b520 0x7fb990148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb994f75700 1 --2- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb990136ad0 0x7fb990094fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fb99005c360 tx=0x7fb988016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb98dffb700 1 -- 172.21.15.23:0/1738705638 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb98800b950 con 0x7fb990136ad0 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb98dffb700 1 -- 172.21.15.23:0/1738705638 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9880205e0 con 0x7fb990136ad0 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb9969d8700 1 -- 172.21.15.23:0/1738705638 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb988009470 con 0x7fb990136ad0 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb98dffb700 1 -- 172.21.15.23:0/1738705638 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb98800b680 con 0x7fb990136ad0 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.614+0000 7fb9969d8700 1 -- 172.21.15.23:0/1738705638 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb990149620 con 0x7fb990136ad0 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.615+0000 7fb98dffb700 1 -- 172.21.15.23:0/1738705638 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7fb98801e070 con 0x7fb990136ad0 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.615+0000 7fb9969d8700 1 -- 172.21.15.23:0/1738705638 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb990057a60 con 0x7fb990136ad0 2024-03-13T21:32:47.609 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.615+0000 7fb98dffb700 1 -- 172.21.15.23:0/1738705638 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7fb98804b140 con 0x7fb990136ad0 2024-03-13T21:32:47.610 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.618+0000 7fb98dffb700 1 -- 172.21.15.23:0/1738705638 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7fb988020ef0 con 0x7fb990136ad0 2024-03-13T21:32:47.725 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.734+0000 7fb9969d8700 1 -- 172.21.15.23:0/1738705638 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb990060880 con 0x7fb990136ad0 2024-03-13T21:32:47.725 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.734+0000 7fb98dffb700 1 -- 172.21.15.23:0/1738705638 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v14) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb988015020 con 0x7fb990136ad0 2024-03-13T21:32:47.725 INFO:teuthology.orchestra.run.smithi023.stdout:pg_num: 8 2024-03-13T21:32:47.727 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.735+0000 7fb9969d8700 1 -- 172.21.15.23:0/1738705638 >> v1:172.21.15.23:6816/39358 conn(0x7fb974038e30 legacy=0x7fb97403b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.727 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.735+0000 7fb9969d8700 1 -- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb990136ad0 msgr2=0x7fb990094fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.727 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.735+0000 7fb9969d8700 1 --2- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb990136ad0 0x7fb990094fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fb99005c360 tx=0x7fb988016000).stop 2024-03-13T21:32:47.727 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.735+0000 7fb9969d8700 1 -- 172.21.15.23:0/1738705638 shutdown_connections 2024-03-13T21:32:47.727 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.735+0000 7fb9969d8700 1 --2- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7fb990136ad0 0x7fb990094fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.727 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.735+0000 7fb9969d8700 1 --2- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7fb99009b520 0x7fb990148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.727 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.735+0000 7fb9969d8700 1 --2- 172.21.15.23:0/1738705638 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7fb9901366f0 0x7fb990093660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.727 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.735+0000 7fb9969d8700 1 -- 172.21.15.23:0/1738705638 >> 172.21.15.23:0/1738705638 conn(0x7fb99006c0c0 msgr2=0x7fb9901350b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:47.727 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.736+0000 7fb9969d8700 1 -- 172.21.15.23:0/1738705638 shutdown_connections 2024-03-13T21:32:47.728 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.736+0000 7fb9969d8700 1 -- 172.21.15.23:0/1738705638 wait complete. 2024-03-13T21:32:47.737 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-13T21:32:47.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.851+0000 7f446dfdb700 1 Processor -- start 2024-03-13T21:32:47.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.851+0000 7f446dfdb700 1 -- start start 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f446dfdb700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f4468132df0 0x7f4468060dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f446dfdb700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f44681331d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f446dfdb700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468067940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f446dfdb700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44680603c0 con 0x7f4468132df0 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f446dfdb700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44680aa230 con 0x7f44680674b0 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f446dfdb700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44680aa4d0 con 0x7f44680aae90 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f4466ffd700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f44681331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f44677fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f4468132df0 0x7f4468060dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f4467fff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468067940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f44677fe700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f4468132df0 0x7f4468060dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:49978/0 (socket says 172.21.15.23:49978) 2024-03-13T21:32:47.845 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f4467fff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468067940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44882/0 (socket says 172.21.15.23:44882) 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f4466ffd700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f44681331d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36444/0 (socket says 172.21.15.23:36444) 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f44677fe700 1 -- 172.21.15.23:0/3717180885 learned_addr learned my addr 172.21.15.23:0/3717180885 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f4466ffd700 1 -- 172.21.15.23:0/3717180885 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 msgr2=0x7f4468067940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f4466ffd700 1 --2- 172.21.15.23:0/3717180885 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468067940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f4466ffd700 1 -- 172.21.15.23:0/3717180885 >> v2:172.21.15.23:3300/0 conn(0x7f4468132df0 msgr2=0x7f4468060dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f4466ffd700 1 --2- 172.21.15.23:0/3717180885 >> v2:172.21.15.23:3300/0 conn(0x7f4468132df0 0x7f4468060dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f4466ffd700 1 -- 172.21.15.23:0/3717180885 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44680aa790 con 0x7f44680aae90 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.852+0000 7f44677fe700 1 --2- 172.21.15.23:0/3717180885 >> v2:172.21.15.23:3300/0 conn(0x7f4468132df0 0x7f4468060dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f4466ffd700 1 --2- 172.21.15.23:0/3717180885 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f44681331d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f4450002f70 tx=0x7f4450006df0).ready entity=mon.2 client_cookie=43b7176643e7418a server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f44667fc700 1 -- 172.21.15.23:0/3717180885 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445000d4a0 con 0x7f44680aae90 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f44667fc700 1 -- 172.21.15.23:0/3717180885 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4450000e90 con 0x7f44680aae90 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f44667fc700 1 -- 172.21.15.23:0/3717180885 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445000d7f0 con 0x7f44680aae90 2024-03-13T21:32:47.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f446dfdb700 1 -- 172.21.15.23:0/3717180885 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 msgr2=0x7f44681331d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f446dfdb700 1 --2- 172.21.15.23:0/3717180885 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f44681331d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f4450002f70 tx=0x7f4450006df0).stop 2024-03-13T21:32:47.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f446dfdb700 1 -- 172.21.15.23:0/3717180885 shutdown_connections 2024-03-13T21:32:47.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f446dfdb700 1 --2- 172.21.15.23:0/3717180885 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468067940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f446dfdb700 1 --2- 172.21.15.23:0/3717180885 >> v2:172.21.15.23:3300/0 conn(0x7f4468132df0 0x7f4468060dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f446dfdb700 1 --2- 172.21.15.23:0/3717180885 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f44681331d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f446dfdb700 1 -- 172.21.15.23:0/3717180885 >> 172.21.15.23:0/3717180885 conn(0x7f446805c300 msgr2=0x7f446806c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:47.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f446dfdb700 1 -- 172.21.15.23:0/3717180885 shutdown_connections 2024-03-13T21:32:47.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.853+0000 7f446dfdb700 1 -- 172.21.15.23:0/3717180885 wait complete. 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.854+0000 7f446dfdb700 1 Processor -- start 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f446dfdb700 1 -- start start 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f446dfdb700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468098b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f446dfdb700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f4468148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f446dfdb700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4468132df0 0x7f4468149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f446dfdb700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44680997c0 con 0x7f4468132df0 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f446dfdb700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4468096280 con 0x7f44680674b0 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f446dfdb700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4468138780 con 0x7f44680aae90 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f44677fe700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468098b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f44677fe700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468098b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:44896/0 (socket says 172.21.15.23:44896) 2024-03-13T21:32:47.850 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f4466ffd700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f4468148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f44677fe700 1 -- 172.21.15.23:0/2004813257 learned_addr learned my addr 172.21.15.23:0/2004813257 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.855+0000 7f4467fff700 1 --2- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4468132df0 0x7f4468149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f44677fe700 1 -- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 msgr2=0x7f4468148b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f44677fe700 1 --2- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f4468148b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f44677fe700 1 -- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4468132df0 msgr2=0x7f4468149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f44677fe700 1 --2- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4468132df0 0x7f4468149060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f44677fe700 1 -- 172.21.15.23:0/2004813257 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445800b1e0 con 0x7f44680674b0 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f4466ffd700 1 --2- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f4468148b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f44677fe700 1 --2- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468098b30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f445800c000 tx=0x7f4458006eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f4464ff9700 1 -- 172.21.15.23:0/2004813257 <== mon.1 v2:172.21.15.23:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4458012600 con 0x7f44680674b0 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f4464ff9700 1 -- 172.21.15.23:0/2004813257 <== mon.1 v2:172.21.15.23:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4458012760 con 0x7f44680674b0 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f4464ff9700 1 -- 172.21.15.23:0/2004813257 <== mon.1 v2:172.21.15.23:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4458012ac0 con 0x7f44680674b0 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f446dfdb700 1 -- 172.21.15.23:0/2004813257 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445000b180 con 0x7f44680674b0 2024-03-13T21:32:47.851 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.856+0000 7f446dfdb700 1 -- 172.21.15.23:0/2004813257 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4468097fc0 con 0x7f44680674b0 2024-03-13T21:32:47.852 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.857+0000 7f4464ff9700 1 -- 172.21.15.23:0/2004813257 <== mon.1 v2:172.21.15.23:3301/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f4458022070 con 0x7f44680674b0 2024-03-13T21:32:47.852 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.857+0000 7f446dfdb700 1 -- 172.21.15.23:0/2004813257 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44680611f0 con 0x7f44680674b0 2024-03-13T21:32:47.852 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.857+0000 7f4464ff9700 1 -- 172.21.15.23:0/2004813257 <== mon.1 v2:172.21.15.23:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f445804a9b0 con 0x7f44680674b0 2024-03-13T21:32:47.852 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.860+0000 7f4464ff9700 1 -- 172.21.15.23:0/2004813257 <== mon.1 v2:172.21.15.23:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f4458033560 con 0x7f44680674b0 2024-03-13T21:32:47.969 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.978+0000 7f446dfdb700 1 -- 172.21.15.23:0/2004813257 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f44680a9a90 con 0x7f44680674b0 2024-03-13T21:32:47.970 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.979+0000 7f4464ff9700 1 -- 172.21.15.23:0/2004813257 <== mon.1 v2:172.21.15.23:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v14) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4458033150 con 0x7f44680674b0 2024-03-13T21:32:47.970 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:47.970 INFO:teuthology.orchestra.run.smithi023.stdout:[ 2024-03-13T21:32:47.970 INFO:teuthology.orchestra.run.smithi023.stdout: { 2024-03-13T21:32:47.970 INFO:teuthology.orchestra.run.smithi023.stdout: "poolnum": 1, 2024-03-13T21:32:47.970 INFO:teuthology.orchestra.run.smithi023.stdout: "poolname": "device_health_metrics" 2024-03-13T21:32:47.970 INFO:teuthology.orchestra.run.smithi023.stdout: }, 2024-03-13T21:32:47.971 INFO:teuthology.orchestra.run.smithi023.stdout: { 2024-03-13T21:32:47.971 INFO:teuthology.orchestra.run.smithi023.stdout: "poolnum": 2, 2024-03-13T21:32:47.971 INFO:teuthology.orchestra.run.smithi023.stdout: "poolname": "rbd" 2024-03-13T21:32:47.971 INFO:teuthology.orchestra.run.smithi023.stdout: } 2024-03-13T21:32:47.971 INFO:teuthology.orchestra.run.smithi023.stdout:] 2024-03-13T21:32:47.972 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.980+0000 7f446dfdb700 1 -- 172.21.15.23:0/2004813257 >> v1:172.21.15.23:6816/39358 conn(0x7f444c038dd0 legacy=0x7f444c03b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.972 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.980+0000 7f446dfdb700 1 -- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 msgr2=0x7f4468098b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:47.972 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.980+0000 7f446dfdb700 1 --2- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468098b30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f445800c000 tx=0x7f4458006eb0).stop 2024-03-13T21:32:47.972 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.980+0000 7f446dfdb700 1 -- 172.21.15.23:0/2004813257 shutdown_connections 2024-03-13T21:32:47.972 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.980+0000 7f446dfdb700 1 --2- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f4468132df0 0x7f4468149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.972 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.980+0000 7f446dfdb700 1 --2- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f44680674b0 0x7f4468098b30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.972 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.980+0000 7f446dfdb700 1 --2- 172.21.15.23:0/2004813257 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f44680aae90 0x7f4468148b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:47.972 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.980+0000 7f446dfdb700 1 -- 172.21.15.23:0/2004813257 >> 172.21.15.23:0/2004813257 conn(0x7f446805c300 msgr2=0x7f4468136750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:47.973 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.980+0000 7f446dfdb700 1 -- 172.21.15.23:0/2004813257 shutdown_connections 2024-03-13T21:32:47.973 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:47.980+0000 7f446dfdb700 1 -- 172.21.15.23:0/2004813257 wait complete. 2024-03-13T21:32:47.981 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-13T21:32:47.982 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-13T21:32:48.091 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.100+0000 7f9f28f6b700 1 Processor -- start 2024-03-13T21:32:48.091 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.100+0000 7f9f28f6b700 1 -- start start 2024-03-13T21:32:48.093 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f28f6b700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f241366f0 0x7f9f2409bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.093 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f28f6b700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f2409b520 0x7f9f24136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.093 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f28f6b700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2409a0c0 con 0x7f9f24099a00 2024-03-13T21:32:48.093 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f28f6b700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2409a380 con 0x7f9f2409b520 2024-03-13T21:32:48.093 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f28f6b700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24135fe0 con 0x7f9f241366f0 2024-03-13T21:32:48.093 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f2259c700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f241366f0 0x7f9f2409bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f22d9d700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f9f24099a00 0x7f9f24139530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:59146/0 (socket says 172.21.15.23:59146) 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f22d9d700 1 -- 172.21.15.23:0/1954092185 learned_addr learned my addr 172.21.15.23:0/1954092185 (peer_addr_for_me v1:172.21.15.23:0/0) 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f21d9b700 1 --2- 172.21.15.23:0/1954092185 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f2409b520 0x7f9f24136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f21d9b700 1 -- 172.21.15.23:0/1954092185 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f241366f0 msgr2=0x7f9f2409bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f21d9b700 1 --2- 172.21.15.23:0/1954092185 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f241366f0 0x7f9f2409bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f21d9b700 1 -- 172.21.15.23:0/1954092185 >> v1:172.21.15.23:6789/0 conn(0x7f9f24099a00 legacy=0x7f9f24139530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f2159a700 1 -- 172.21.15.23:0/1954092185 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712026858 0 0) 0x7f9f2409a0c0 con 0x7f9f24099a00 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.101+0000 7f9f21d9b700 1 -- 172.21.15.23:0/1954092185 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f24067cb0 con 0x7f9f2409b520 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f21d9b700 1 --2- 172.21.15.23:0/1954092185 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f2409b520 0x7f9f24136ad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f9f2405c360 tx=0x7f9f14006d90).ready entity=mon.1 client_cookie=81603744221a15e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f2159a700 1 -- 172.21.15.23:0/1954092185 <== mon.1 v2:172.21.15.23:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f1400d4a0 con 0x7f9f2409b520 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f2159a700 1 -- 172.21.15.23:0/1954092185 <== mon.1 v2:172.21.15.23:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14000e90 con 0x7f9f2409b520 2024-03-13T21:32:48.094 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f2159a700 1 -- 172.21.15.23:0/1954092185 <== mon.1 v2:172.21.15.23:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f1400d7f0 con 0x7f9f2409b520 2024-03-13T21:32:48.095 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f28f6b700 1 -- 172.21.15.23:0/1954092185 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f2409b520 msgr2=0x7f9f24136ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:48.095 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f28f6b700 1 --2- 172.21.15.23:0/1954092185 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f2409b520 0x7f9f24136ad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f9f2405c360 tx=0x7f9f14006d90).stop 2024-03-13T21:32:48.095 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f28f6b700 1 -- 172.21.15.23:0/1954092185 shutdown_connections 2024-03-13T21:32:48.095 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f28f6b700 1 --2- 172.21.15.23:0/1954092185 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f241366f0 0x7f9f2409bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.095 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f28f6b700 1 --2- 172.21.15.23:0/1954092185 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f2409b520 0x7f9f24136ad0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.095 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f28f6b700 1 -- 172.21.15.23:0/1954092185 >> 172.21.15.23:0/1954092185 conn(0x7f9f2406c0c0 msgr2=0x7f9f2406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:48.095 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f28f6b700 1 -- 172.21.15.23:0/1954092185 shutdown_connections 2024-03-13T21:32:48.095 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.102+0000 7f9f28f6b700 1 -- 172.21.15.23:0/1954092185 wait complete. 2024-03-13T21:32:48.097 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.103+0000 7f9f28f6b700 1 Processor -- start 2024-03-13T21:32:48.097 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.103+0000 7f9f28f6b700 1 -- start start 2024-03-13T21:32:48.097 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f28f6b700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f24099a00 0x7f9f2412e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f28f6b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9f2409b520 0x7f9f2412eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f28f6b700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f241366f0 0x7f9f2412f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f28f6b700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2412f8c0 con 0x7f9f2409b520 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f28f6b700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24139d80 con 0x7f9f241366f0 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f28f6b700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240936d0 con 0x7f9f24099a00 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f2259c700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f24099a00 0x7f9f2412e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f2259c700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f24099a00 0x7f9f2412e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:36482/0 (socket says 172.21.15.23:36482) 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f2259c700 1 -- 172.21.15.23:0/3723871791 learned_addr learned my addr 172.21.15.23:0/3723871791 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f22d9d700 1 --2- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f241366f0 0x7f9f2412f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f21d9b700 1 --2- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9f2409b520 0x7f9f2412eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f2259c700 1 -- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f241366f0 msgr2=0x7f9f2412f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f2259c700 1 --2- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f241366f0 0x7f9f2412f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f2259c700 1 -- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9f2409b520 msgr2=0x7f9f2412eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f2259c700 1 --2- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9f2409b520 0x7f9f2412eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.098 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f2259c700 1 -- 172.21.15.23:0/3723871791 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1400b120 con 0x7f9f24099a00 2024-03-13T21:32:48.099 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f2259c700 1 --2- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f24099a00 0x7f9f2412e9f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f9f24096750 tx=0x7f9f18006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:48.099 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f137fe700 1 -- 172.21.15.23:0/3723871791 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f18010780 con 0x7f9f24099a00 2024-03-13T21:32:48.099 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f137fe700 1 -- 172.21.15.23:0/3723871791 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f180108e0 con 0x7f9f24099a00 2024-03-13T21:32:48.099 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f28f6b700 1 -- 172.21.15.23:0/3723871791 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f24094150 con 0x7f9f24099a00 2024-03-13T21:32:48.099 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f137fe700 1 -- 172.21.15.23:0/3723871791 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f18010c40 con 0x7f9f24099a00 2024-03-13T21:32:48.099 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.104+0000 7f9f28f6b700 1 -- 172.21.15.23:0/3723871791 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f24095300 con 0x7f9f24099a00 2024-03-13T21:32:48.099 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.105+0000 7f9f137fe700 1 -- 172.21.15.23:0/3723871791 <== mon.2 v2:172.21.15.23:3302/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f9f1801c460 con 0x7f9f24099a00 2024-03-13T21:32:48.099 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.106+0000 7f9f137fe700 1 -- 172.21.15.23:0/3723871791 <== mon.2 v2:172.21.15.23:3302/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2822+0+0 (secure 0 0 0) 0x7f9f1804a810 con 0x7f9f24099a00 2024-03-13T21:32:48.099 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.106+0000 7f9f28f6b700 1 -- 172.21.15.23:0/3723871791 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f24095de0 con 0x7f9f24099a00 2024-03-13T21:32:48.099 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.109+0000 7f9f137fe700 1 -- 172.21.15.23:0/3723871791 <== mon.2 v2:172.21.15.23:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f9f18033490 con 0x7f9f24099a00 2024-03-13T21:32:48.215 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.225+0000 7f9f28f6b700 1 -- 172.21.15.23:0/3723871791 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9f24094410 con 0x7f9f24099a00 2024-03-13T21:32:48.346 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.355+0000 7f9f137fe700 1 -- 172.21.15.23:0/3723871791 <== mon.2 v2:172.21.15.23:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v15) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9f18033080 con 0x7f9f24099a00 2024-03-13T21:32:48.350 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.359+0000 7f9f28f6b700 1 -- 172.21.15.23:0/3723871791 >> v1:172.21.15.23:6816/39358 conn(0x7f9f00038dd0 legacy=0x7f9f0003b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:48.350 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.360+0000 7f9f28f6b700 1 -- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f24099a00 msgr2=0x7f9f2412e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:48.350 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.360+0000 7f9f28f6b700 1 --2- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f24099a00 0x7f9f2412e9f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f9f24096750 tx=0x7f9f18006df0).stop 2024-03-13T21:32:48.351 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.360+0000 7f9f28f6b700 1 -- 172.21.15.23:0/3723871791 shutdown_connections 2024-03-13T21:32:48.351 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.360+0000 7f9f28f6b700 1 --2- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f9f241366f0 0x7f9f2412f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.351 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.360+0000 7f9f28f6b700 1 --2- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f9f24099a00 0x7f9f2412e9f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.351 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.360+0000 7f9f28f6b700 1 --2- 172.21.15.23:0/3723871791 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f9f2409b520 0x7f9f2412eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.351 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.360+0000 7f9f28f6b700 1 -- 172.21.15.23:0/3723871791 >> 172.21.15.23:0/3723871791 conn(0x7f9f2406c0c0 msgr2=0x7f9f240a7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:48.351 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.360+0000 7f9f28f6b700 1 -- 172.21.15.23:0/3723871791 shutdown_connections 2024-03-13T21:32:48.352 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.361+0000 7f9f28f6b700 1 -- 172.21.15.23:0/3723871791 wait complete. 2024-03-13T21:32:48.354 INFO:teuthology.orchestra.run.smithi023.stderr:pool 'cephfs_metadata' created 2024-03-13T21:32:48.363 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-13T21:32:48.475 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.484+0000 7f833b73b700 1 Processor -- start 2024-03-13T21:32:48.475 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.484+0000 7f833b73b700 1 -- start start 2024-03-13T21:32:48.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f833b73b700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f83341366f0 0x7f833409bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f833b73b700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 0x7f8334136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f833b73b700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f8334099a00 0x7f8334139530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f833b73b700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833409a380 con 0x7f83341366f0 2024-03-13T21:32:48.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f833b73b700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8334135fe0 con 0x7f833409b520 2024-03-13T21:32:48.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f833b73b700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8334099e90 con 0x7f8334099a00 2024-03-13T21:32:48.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f83394d7700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f83341366f0 0x7f833409bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f8339cd8700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f8334099a00 0x7f8334139530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f8338cd6700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 0x7f8334136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.478 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f83394d7700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f83341366f0 0x7f833409bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:50810/0 (socket says 172.21.15.23:50810) 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f83394d7700 1 -- 172.21.15.23:0/756323342 learned_addr learned my addr 172.21.15.23:0/756323342 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f8339cd8700 1 -- 172.21.15.23:0/756323342 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 msgr2=0x7f8334136ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f8339cd8700 1 --2- 172.21.15.23:0/756323342 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 0x7f8334136ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f8339cd8700 1 -- 172.21.15.23:0/756323342 >> v2:172.21.15.23:3300/0 conn(0x7f83341366f0 msgr2=0x7f833409bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f8339cd8700 1 --2- 172.21.15.23:0/756323342 >> v2:172.21.15.23:3300/0 conn(0x7f83341366f0 0x7f833409bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f8339cd8700 1 -- 172.21.15.23:0/756323342 --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833409a0c0 con 0x7f8334099a00 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.485+0000 7f83394d7700 1 --2- 172.21.15.23:0/756323342 >> v2:172.21.15.23:3300/0 conn(0x7f83341366f0 0x7f833409bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f8339cd8700 1 --2- 172.21.15.23:0/756323342 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f8334099a00 0x7f8334139530 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f8330002f70 tx=0x7f8330006df0).ready entity=mon.2 client_cookie=d340dcfb41d7fc62 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f8323fff700 1 -- 172.21.15.23:0/756323342 <== mon.2 v2:172.21.15.23:3302/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f833000d4a0 con 0x7f8334099a00 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f8323fff700 1 -- 172.21.15.23:0/756323342 <== mon.2 v2:172.21.15.23:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8330000e90 con 0x7f8334099a00 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f8323fff700 1 -- 172.21.15.23:0/756323342 <== mon.2 v2:172.21.15.23:3302/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f833000d7f0 con 0x7f8334099a00 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f833b73b700 1 -- 172.21.15.23:0/756323342 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f8334099a00 msgr2=0x7f8334139530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f833b73b700 1 --2- 172.21.15.23:0/756323342 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f8334099a00 0x7f8334139530 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f8330002f70 tx=0x7f8330006df0).stop 2024-03-13T21:32:48.479 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f833b73b700 1 -- 172.21.15.23:0/756323342 shutdown_connections 2024-03-13T21:32:48.480 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f833b73b700 1 --2- 172.21.15.23:0/756323342 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f8334099a00 0x7f8334139530 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.480 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f833b73b700 1 --2- 172.21.15.23:0/756323342 >> v2:172.21.15.23:3300/0 conn(0x7f83341366f0 0x7f833409bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.480 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f833b73b700 1 --2- 172.21.15.23:0/756323342 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 0x7f8334136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.480 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f833b73b700 1 -- 172.21.15.23:0/756323342 >> 172.21.15.23:0/756323342 conn(0x7f833406c0c0 msgr2=0x7f833406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:48.480 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f833b73b700 1 -- 172.21.15.23:0/756323342 shutdown_connections 2024-03-13T21:32:48.480 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.486+0000 7f833b73b700 1 -- 172.21.15.23:0/756323342 wait complete. 2024-03-13T21:32:48.480 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.487+0000 7f833b73b700 1 Processor -- start 2024-03-13T21:32:48.480 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.487+0000 7f833b73b700 1 -- start start 2024-03-13T21:32:48.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f833b73b700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8334099a00 0x7f8334097650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f833b73b700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 0x7f8334092fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f833b73b700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f83341366f0 0x7f83340934b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:48.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f833b73b700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8334095380 con 0x7f8334099a00 2024-03-13T21:32:48.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f833b73b700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8334096970 con 0x7f833409b520 2024-03-13T21:32:48.482 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f833b73b700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8334095e90 con 0x7f83341366f0 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f8339cd8700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f83341366f0 0x7f83340934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f83394d7700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8334099a00 0x7f8334097650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f8338cd6700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 0x7f8334092fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f8339cd8700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f83341366f0 0x7f83340934b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:33438/0 (socket says 172.21.15.23:33438) 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f8339cd8700 1 -- 172.21.15.23:0/3683120014 learned_addr learned my addr 172.21.15.23:0/3683120014 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f83394d7700 1 -- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f83341366f0 msgr2=0x7f83340934b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f83394d7700 1 --2- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f83341366f0 0x7f83340934b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f83394d7700 1 -- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 msgr2=0x7f8334092fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f83394d7700 1 --2- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 0x7f8334092fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f83394d7700 1 -- 172.21.15.23:0/3683120014 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c009530 con 0x7f8334099a00 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f8339cd8700 1 --2- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f83341366f0 0x7f83340934b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f8338cd6700 1 --2- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 0x7f8334092fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:48.483 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f83394d7700 1 --2- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8334099a00 0x7f8334097650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f832c00c000 tx=0x7f832c009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:48.484 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f83227fc700 1 -- 172.21.15.23:0/3683120014 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f832c00c850 con 0x7f8334099a00 2024-03-13T21:32:48.484 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f83227fc700 1 -- 172.21.15.23:0/3683120014 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c00c9b0 con 0x7f8334099a00 2024-03-13T21:32:48.484 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f833b73b700 1 -- 172.21.15.23:0/3683120014 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833000b180 con 0x7f8334099a00 2024-03-13T21:32:48.484 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.488+0000 7f83227fc700 1 -- 172.21.15.23:0/3683120014 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f832c00cd10 con 0x7f8334099a00 2024-03-13T21:32:48.484 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.489+0000 7f833b73b700 1 -- 172.21.15.23:0/3683120014 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83340963d0 con 0x7f8334099a00 2024-03-13T21:32:48.484 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.490+0000 7f833b73b700 1 -- 172.21.15.23:0/3683120014 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8334096680 con 0x7f8334099a00 2024-03-13T21:32:48.484 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.490+0000 7f83227fc700 1 -- 172.21.15.23:0/3683120014 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f832c0123f0 con 0x7f8334099a00 2024-03-13T21:32:48.484 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.490+0000 7f83227fc700 1 -- 172.21.15.23:0/3683120014 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3192+0+0 (secure 0 0 0) 0x7f832c04a8e0 con 0x7f8334099a00 2024-03-13T21:32:48.484 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.493+0000 7f83227fc700 1 -- 172.21.15.23:0/3683120014 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f832c02f9c0 con 0x7f8334099a00 2024-03-13T21:32:48.600 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:48.609+0000 7f833b73b700 1 -- 172.21.15.23:0/3683120014 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f83340990d0 con 0x7f8334099a00 2024-03-13T21:32:49.371 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.380+0000 7f83227fc700 1 -- 172.21.15.23:0/3683120014 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v16) v1 ==== 170+0+0 (secure 0 0 0) 0x7f832c032fe0 con 0x7f8334099a00 2024-03-13T21:32:49.374 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.383+0000 7f833b73b700 1 -- 172.21.15.23:0/3683120014 >> v1:172.21.15.23:6816/39358 conn(0x7f8314038da0 legacy=0x7f831403b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.374 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.383+0000 7f833b73b700 1 -- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8334099a00 msgr2=0x7f8334097650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.374 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.383+0000 7f833b73b700 1 --2- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8334099a00 0x7f8334097650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f832c00c000 tx=0x7f832c009290).stop 2024-03-13T21:32:49.374 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.383+0000 7f833b73b700 1 -- 172.21.15.23:0/3683120014 shutdown_connections 2024-03-13T21:32:49.374 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.383+0000 7f833b73b700 1 --2- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f83341366f0 0x7f83340934b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.374 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.383+0000 7f833b73b700 1 --2- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f8334099a00 0x7f8334097650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.374 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.383+0000 7f833b73b700 1 --2- 172.21.15.23:0/3683120014 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f833409b520 0x7f8334092fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.375 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.383+0000 7f833b73b700 1 -- 172.21.15.23:0/3683120014 >> 172.21.15.23:0/3683120014 conn(0x7f833406c0c0 msgr2=0x7f83340663e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:49.375 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.383+0000 7f833b73b700 1 -- 172.21.15.23:0/3683120014 shutdown_connections 2024-03-13T21:32:49.375 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.383+0000 7f833b73b700 1 -- 172.21.15.23:0/3683120014 wait complete. 2024-03-13T21:32:49.375 INFO:teuthology.orchestra.run.smithi023.stderr:pool 'cephfs_data' created 2024-03-13T21:32:49.402 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-13T21:32:49.522 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.531+0000 7f1bc7475700 1 Processor -- start 2024-03-13T21:32:49.522 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.531+0000 7f1bc7475700 1 -- start start 2024-03-13T21:32:49.524 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc7475700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc01366f0 0x7f1bc009bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.524 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc7475700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f1bc009b520 0x7f1bc0136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.524 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc7475700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc0099a00 0x7f1bc0139530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.524 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc7475700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc009a380 con 0x7f1bc009b520 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc7475700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc0135fe0 con 0x7f1bc0099a00 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc7475700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc0099eb0 con 0x7f1bc01366f0 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5211700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc01366f0 0x7f1bc009bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5a12700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc0099a00 0x7f1bc0139530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc4a10700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f1bc009b520 0x7f1bc0136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5211700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc01366f0 0x7f1bc009bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:33450/0 (socket says 172.21.15.23:33450) 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc4a10700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f1bc009b520 0x7f1bc0136ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:50822/0 (socket says 172.21.15.23:50822) 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5a12700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc0099a00 0x7f1bc0139530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:56324/0 (socket says 172.21.15.23:56324) 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5211700 1 -- 172.21.15.23:0/3611021249 learned_addr learned my addr 172.21.15.23:0/3611021249 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5a12700 1 -- 172.21.15.23:0/3611021249 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc01366f0 msgr2=0x7f1bc009bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5a12700 1 --2- 172.21.15.23:0/3611021249 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc01366f0 0x7f1bc009bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5a12700 1 -- 172.21.15.23:0/3611021249 >> v2:172.21.15.23:3300/0 conn(0x7f1bc009b520 msgr2=0x7f1bc0136ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:49.525 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5a12700 1 --2- 172.21.15.23:0/3611021249 >> v2:172.21.15.23:3300/0 conn(0x7f1bc009b520 0x7f1bc0136ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5a12700 1 -- 172.21.15.23:0/3611021249 --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc009a0c0 con 0x7f1bc0099a00 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5211700 1 --2- 172.21.15.23:0/3611021249 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc01366f0 0x7f1bc009bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.532+0000 7f1bc5a12700 1 --2- 172.21.15.23:0/3611021249 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc0099a00 0x7f1bc0139530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f1bb80032f0 tx=0x7f1bb80091d0).ready entity=mon.1 client_cookie=29cabc30d2e7e5af server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1baffff700 1 -- 172.21.15.23:0/3611021249 <== mon.1 v2:172.21.15.23:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bb800b600 con 0x7f1bc0099a00 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1baffff700 1 -- 172.21.15.23:0/3611021249 <== mon.1 v2:172.21.15.23:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb800b760 con 0x7f1bc0099a00 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1baffff700 1 -- 172.21.15.23:0/3611021249 <== mon.1 v2:172.21.15.23:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bb800b8d0 con 0x7f1bc0099a00 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1bc7475700 1 -- 172.21.15.23:0/3611021249 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc0099a00 msgr2=0x7f1bc0139530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1bc7475700 1 --2- 172.21.15.23:0/3611021249 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc0099a00 0x7f1bc0139530 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f1bb80032f0 tx=0x7f1bb80091d0).stop 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1bc7475700 1 -- 172.21.15.23:0/3611021249 shutdown_connections 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1bc7475700 1 --2- 172.21.15.23:0/3611021249 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc0099a00 0x7f1bc0139530 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1bc7475700 1 --2- 172.21.15.23:0/3611021249 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc01366f0 0x7f1bc009bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.526 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1bc7475700 1 --2- 172.21.15.23:0/3611021249 >> v2:172.21.15.23:3300/0 conn(0x7f1bc009b520 0x7f1bc0136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.527 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1bc7475700 1 -- 172.21.15.23:0/3611021249 >> 172.21.15.23:0/3611021249 conn(0x7f1bc006c0c0 msgr2=0x7f1bc006c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:49.527 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1bc7475700 1 -- 172.21.15.23:0/3611021249 shutdown_connections 2024-03-13T21:32:49.527 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.533+0000 7f1bc7475700 1 -- 172.21.15.23:0/3611021249 wait complete. 2024-03-13T21:32:49.529 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.534+0000 7f1bc7475700 1 Processor -- start 2024-03-13T21:32:49.529 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.534+0000 7f1bc7475700 1 -- start start 2024-03-13T21:32:49.529 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc7475700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f1bc0099a00 0x7f1bc012ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.529 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc7475700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc009b520 0x7f1bc012f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.529 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc7475700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc01366f0 0x7f1bc0145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.529 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc7475700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc0095680 con 0x7f1bc0099a00 2024-03-13T21:32:49.529 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc7475700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc0095b80 con 0x7f1bc01366f0 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc7475700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc012f830 con 0x7f1bc009b520 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc4a10700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc009b520 0x7f1bc012f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5211700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f1bc0099a00 0x7f1bc012ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5211700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f1bc0099a00 0x7f1bc012ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:50834/0 (socket says 172.21.15.23:50834) 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc4a10700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc009b520 0x7f1bc012f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:33462/0 (socket says 172.21.15.23:33462) 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5211700 1 -- 172.21.15.23:0/4229848023 learned_addr learned my addr 172.21.15.23:0/4229848023 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5a12700 1 --2- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc01366f0 0x7f1bc0145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5211700 1 -- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc009b520 msgr2=0x7f1bc012f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5211700 1 --2- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc009b520 0x7f1bc012f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5211700 1 -- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc01366f0 msgr2=0x7f1bc0145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.530 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5211700 1 --2- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc01366f0 0x7f1bc0145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5211700 1 -- 172.21.15.23:0/4229848023 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc00b1e0 con 0x7f1bc0099a00 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc4a10700 1 --2- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc009b520 0x7f1bc012f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5a12700 1 --2- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc01366f0 0x7f1bc0145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc5211700 1 --2- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f1bc0099a00 0x7f1bc012ee50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f1bc012f9c0 tx=0x7f1bbc006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bae7fc700 1 -- 172.21.15.23:0/4229848023 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bbc012600 con 0x7f1bc0099a00 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bae7fc700 1 -- 172.21.15.23:0/4229848023 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc012760 con 0x7f1bc0099a00 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bc7475700 1 -- 172.21.15.23:0/4229848023 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb8009470 con 0x7f1bc0099a00 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.535+0000 7f1bae7fc700 1 -- 172.21.15.23:0/4229848023 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bbc012ac0 con 0x7f1bc0099a00 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.536+0000 7f1bc7475700 1 -- 172.21.15.23:0/4229848023 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc00943b0 con 0x7f1bc0099a00 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.537+0000 7f1bc7475700 1 -- 172.21.15.23:0/4229848023 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc0057a60 con 0x7f1bc0099a00 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.537+0000 7f1bae7fc700 1 -- 172.21.15.23:0/4229848023 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f1bbc022070 con 0x7f1bc0099a00 2024-03-13T21:32:49.531 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.537+0000 7f1bae7fc700 1 -- 172.21.15.23:0/4229848023 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3574+0+0 (secure 0 0 0) 0x7f1bbc04ac90 con 0x7f1bc0099a00 2024-03-13T21:32:49.532 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.540+0000 7f1bae7fc700 1 -- 172.21.15.23:0/4229848023 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f1bbc033580 con 0x7f1bc0099a00 2024-03-13T21:32:49.680 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.689+0000 7f1bc7475700 1 -- 172.21.15.23:0/4229848023 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1bc00a7ba0 con 0x7f1bc0099a00 2024-03-13T21:32:49.687 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.696+0000 7f1bae7fc700 1 -- 172.21.15.23:0/4229848023 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v5) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1bbc033170 con 0x7f1bc0099a00 2024-03-13T21:32:49.702 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.712+0000 7f1bc7475700 1 -- 172.21.15.23:0/4229848023 >> v1:172.21.15.23:6816/39358 conn(0x7f1ba0038da0 legacy=0x7f1ba003b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.703 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.712+0000 7f1bc7475700 1 -- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f1bc0099a00 msgr2=0x7f1bc012ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.703 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.712+0000 7f1bc7475700 1 --2- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f1bc0099a00 0x7f1bc012ee50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f1bc012f9c0 tx=0x7f1bbc006eb0).stop 2024-03-13T21:32:49.708 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.717+0000 7f1bc7475700 1 -- 172.21.15.23:0/4229848023 shutdown_connections 2024-03-13T21:32:49.708 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.717+0000 7f1bc7475700 1 --2- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f1bc01366f0 0x7f1bc0145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.708 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.717+0000 7f1bc7475700 1 --2- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f1bc0099a00 0x7f1bc012ee50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.708 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.717+0000 7f1bc7475700 1 --2- 172.21.15.23:0/4229848023 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f1bc009b520 0x7f1bc012f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.708 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.717+0000 7f1bc7475700 1 -- 172.21.15.23:0/4229848023 >> 172.21.15.23:0/4229848023 conn(0x7f1bc006c0c0 msgr2=0x7f1bc0066810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:49.709 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.717+0000 7f1bc7475700 1 -- 172.21.15.23:0/4229848023 shutdown_connections 2024-03-13T21:32:49.709 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.718+0000 7f1bc7475700 1 -- 172.21.15.23:0/4229848023 wait complete. 2024-03-13T21:32:49.714 INFO:teuthology.orchestra.run.smithi023.stderr:new fs with metadata pool 3 and data pool 4 2024-03-13T21:32:49.733 DEBUG:teuthology.orchestra.run.smithi023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T21:32:49.839 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.848+0000 7f6a2afcf700 1 Processor -- start 2024-03-13T21:32:49.839 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.848+0000 7f6a2afcf700 1 -- start start 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2afcf700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 0x7f6a240621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2afcf700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f6a24136140 0x7f6a2409adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2afcf700 1 -- --> v1:172.21.15.23:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a2409bc50 con 0x7f6a2409aa10 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2afcf700 1 -- --> v2:172.21.15.23:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2409b2e0 con 0x7f6a24136140 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2afcf700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241365d0 con 0x7f6a24064700 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2956c700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f6a24136140 0x7f6a2409adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a23fff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 0x7f6a240621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2956c700 1 --2- >> v2:172.21.15.23:3300/0 conn(0x7f6a24136140 0x7f6a2409adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:50836/0 (socket says 172.21.15.23:50836) 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a28d6b700 1 --1- >> v1:172.21.15.23:6789/0 conn(0x7f6a2409aa10 0x7f6a24132c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.23:6789/0 says I am v1:172.21.15.23:48560/0 (socket says 172.21.15.23:48560) 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2956c700 1 -- 172.21.15.23:0/63542420 learned_addr learned my addr 172.21.15.23:0/63542420 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a23fff700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 0x7f6a240621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3301/0 says I am v2:172.21.15.23:56340/0 (socket says 172.21.15.23:56340) 2024-03-13T21:32:49.842 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2956c700 1 -- 172.21.15.23:0/63542420 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 msgr2=0x7f6a240621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-13T21:32:49.843 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2956c700 1 --2- 172.21.15.23:0/63542420 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 0x7f6a240621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.843 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2956c700 1 -- 172.21.15.23:0/63542420 >> v1:172.21.15.23:6789/0 conn(0x7f6a2409aa10 legacy=0x7f6a24132c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.843 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2956c700 1 -- 172.21.15.23:0/63542420 --> v2:172.21.15.23:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a241367e0 con 0x7f6a24136140 2024-03-13T21:32:49.843 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a237fe700 1 -- 172.21.15.23:0/63542420 <== mon.0 v1:172.21.15.23:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356869257 0 0) 0x7f6a2409bc50 con 0x7f6a2409aa10 2024-03-13T21:32:49.843 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a23fff700 1 --2- 172.21.15.23:0/63542420 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 0x7f6a240621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-03-13T21:32:49.843 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.849+0000 7f6a2956c700 1 --2- 172.21.15.23:0/63542420 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f6a24136140 0x7f6a2409adf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6a1c0032f0 tx=0x7f6a1c009290).ready entity=mon.0 client_cookie=9d70e6764896c2a server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:49.843 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a237fe700 1 -- 172.21.15.23:0/63542420 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a1c00b8a0 con 0x7f6a24136140 2024-03-13T21:32:49.843 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a237fe700 1 -- 172.21.15.23:0/63542420 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c00ba00 con 0x7f6a24136140 2024-03-13T21:32:49.843 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a237fe700 1 -- 172.21.15.23:0/63542420 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a1c00bb70 con 0x7f6a24136140 2024-03-13T21:32:49.843 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a2afcf700 1 -- 172.21.15.23:0/63542420 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f6a24136140 msgr2=0x7f6a2409adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.844 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a2afcf700 1 --2- 172.21.15.23:0/63542420 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f6a24136140 0x7f6a2409adf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6a1c0032f0 tx=0x7f6a1c009290).stop 2024-03-13T21:32:49.844 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a2afcf700 1 -- 172.21.15.23:0/63542420 shutdown_connections 2024-03-13T21:32:49.844 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a2afcf700 1 --2- 172.21.15.23:0/63542420 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f6a24136140 0x7f6a2409adf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.844 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a2afcf700 1 --2- 172.21.15.23:0/63542420 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 0x7f6a240621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.844 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a2afcf700 1 -- 172.21.15.23:0/63542420 >> 172.21.15.23:0/63542420 conn(0x7f6a2406c0c0 msgr2=0x7f6a2406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T21:32:49.844 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a2afcf700 1 -- 172.21.15.23:0/63542420 shutdown_connections 2024-03-13T21:32:49.844 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.850+0000 7f6a2afcf700 1 -- 172.21.15.23:0/63542420 wait complete. 2024-03-13T21:32:49.844 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.851+0000 7f6a2afcf700 1 Processor -- start 2024-03-13T21:32:49.844 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.851+0000 7f6a2afcf700 1 -- start start 2024-03-13T21:32:49.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2afcf700 1 --2- >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 0x7f6a240961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2afcf700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f6a2409aa10 0x7f6a24097ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2afcf700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f6a24136140 0x7f6a2405b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-03-13T21:32:49.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2afcf700 1 -- --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24098ca0 con 0x7f6a24136140 2024-03-13T21:32:49.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2afcf700 1 -- --> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24094cc0 con 0x7f6a24064700 2024-03-13T21:32:49.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2afcf700 1 -- --> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a24096f00 con 0x7f6a2409aa10 2024-03-13T21:32:49.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a23fff700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f6a2409aa10 0x7f6a24097ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2956c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f6a24136140 0x7f6a2405b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.846 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a23fff700 1 --2- >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f6a2409aa10 0x7f6a24097ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3302/0 says I am v2:172.21.15.23:33472/0 (socket says 172.21.15.23:33472) 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2956c700 1 --2- >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f6a24136140 0x7f6a2405b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.23:3300/0 says I am v2:172.21.15.23:50848/0 (socket says 172.21.15.23:50848) 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a23fff700 1 -- 172.21.15.23:0/4255339355 learned_addr learned my addr 172.21.15.23:0/4255339355 (peer_addr_for_me v2:172.21.15.23:0/0) 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a28d6b700 1 --2- 172.21.15.23:0/4255339355 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 0x7f6a240961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2956c700 1 -- 172.21.15.23:0/4255339355 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f6a2409aa10 msgr2=0x7f6a24097ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2956c700 1 --2- 172.21.15.23:0/4255339355 >> [v2:172.21.15.23:3302/0,v1:172.21.15.23:6791/0] conn(0x7f6a2409aa10 0x7f6a24097ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2956c700 1 -- 172.21.15.23:0/4255339355 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 msgr2=0x7f6a240961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2956c700 1 --2- 172.21.15.23:0/4255339355 >> [v2:172.21.15.23:3301/0,v1:172.21.15.23:6790/0] conn(0x7f6a24064700 0x7f6a240961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2956c700 1 -- 172.21.15.23:0/4255339355 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1000b120 con 0x7f6a24136140 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a2956c700 1 --2- 172.21.15.23:0/4255339355 >> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] conn(0x7f6a24136140 0x7f6a2405b970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f6a1c0032f0 tx=0x7f6a1c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.852+0000 7f6a21ffb700 1 -- 172.21.15.23:0/4255339355 <== mon.0 v2:172.21.15.23:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a1c00bb70 con 0x7f6a24136140 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.853+0000 7f6a21ffb700 1 -- 172.21.15.23:0/4255339355 <== mon.0 v2:172.21.15.23:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c021630 con 0x7f6a24136140 2024-03-13T21:32:49.847 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.853+0000 7f6a21ffb700 1 -- 172.21.15.23:0/4255339355 <== mon.0 v2:172.21.15.23:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a1c00b8a0 con 0x7f6a24136140 2024-03-13T21:32:49.848 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.853+0000 7f6a2afcf700 1 -- 172.21.15.23:0/4255339355 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c009530 con 0x7f6a24136140 2024-03-13T21:32:49.848 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.853+0000 7f6a2afcf700 1 -- 172.21.15.23:0/4255339355 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a24097930 con 0x7f6a24136140 2024-03-13T21:32:49.848 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.853+0000 7f6a21ffb700 1 -- 172.21.15.23:0/4255339355 <== mon.0 v2:172.21.15.23:3300/0 4 ==== mgrmap(e 5) v1 ==== 42017+0+0 (secure 0 0 0) 0x7f6a1c028070 con 0x7f6a24136140 2024-03-13T21:32:49.848 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.853+0000 7f6a2afcf700 1 -- 172.21.15.23:0/4255339355 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a24097b50 con 0x7f6a24136140 2024-03-13T21:32:49.848 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.854+0000 7f6a21ffb700 1 -- 172.21.15.23:0/4255339355 <== mon.0 v2:172.21.15.23:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3674+0+0 (secure 0 0 0) 0x7f6a1c04b6b0 con 0x7f6a24136140 2024-03-13T21:32:49.848 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.857+0000 7f6a21ffb700 1 -- 172.21.15.23:0/4255339355 <== mon.0 v2:172.21.15.23:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+108456 (secure 0 0 0) 0x7f6a24097b50 con 0x7f6a24136140 2024-03-13T21:32:49.965 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.974+0000 7f6a2afcf700 1 -- 172.21.15.23:0/4255339355 --> [v2:172.21.15.23:3300/0,v1:172.21.15.23:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a24094e30 con 0x7f6a24136140 2024-03-13T21:32:49.965 INFO:teuthology.orchestra.run.smithi023.stderr:2024-03-13T21:32:49.975+0000 7f6a21ffb700 1 -- 172.21.15.23:0/4255339355 <== mon.0 v2:172.21.15.23:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+10545 (secure 0 0 0) 0x7f6a24094e30 con 0x7f6a24136140 2024-03-13T21:32:49.966 INFO:teuthology.orchestra.run.smithi023.stdout: 2024-03-13T21:32:49.966 INFO:teuthology.orchestra.run.smithi023.stdout:{"epoch":17,"fsid":"19813cba-5ddc-4ec3-85b1-5e37baeea636","created":"2024-03-13T21:32:30.539980+0000","modified":"2024-03-13T21:32:49.691167+0000","last_up_change":"2024-03-13T21:32:36.292795+0000","last_in_change":"2024-03-13T21:32:36.292795+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-03-13T21:32:33.319887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-13T21:32:39.554513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-03-13T21:32:48.270717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-03-13T21:32:48.654626+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version"