2024-09-15T21:44:48.121 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T21:44:48.142 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:44:48.239 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906220 branch: squid description: fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906220' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'install ceph/quincy latest ' name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on osd down out interval zero: false mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request sha1: 8921c8ef947807b717db041d85f0761b7914d5fc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDnmgv8AOVyOcHSLuNx7yek9GMfWCx0EyjgIsM38xbzh6WhkY5WqoA0n0FTmFcLOOaeEadFQy6lh35lya6L80f8= tasks: - install: branch: quincy exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-rook - ceph-mgr-cephadm - cephadm - ceph-volume extra_packages: - librados2 - print: '**** done installing quincy' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client quincy - print: '**** done ceph' - print: '*** upgrading, no cephfs present' - exec: mon.a: - ceph fs dump - install.upgrade: mon.a: null - print: '**** done install.upgrade' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph versions - ceph osd dump -f json-pretty - ceph fs dump - ceph osd require-osd-release squid - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph.healthy: null - print: '**** done ceph.restart' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T21:44:48.239 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-15T21:44:48.240 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-15T21:44:48.240 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T21:44:48.242 INFO:teuthology.task.internal:Checking packages... 2024-09-15T21:44:48.263 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-15T21:44:48.263 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:44:48.263 INFO:teuthology.packaging:ref: None 2024-09-15T21:44:48.263 INFO:teuthology.packaging:tag: None 2024-09-15T21:44:48.263 INFO:teuthology.packaging:branch: squid 2024-09-15T21:44:48.263 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:44:48.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T21:44:48.460 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-326.g8921c8ef 2024-09-15T21:44:48.462 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T21:44:48.468 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T21:44:48.468 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T21:44:48.488 INFO:teuthology.task.internal:Saving configuration 2024-09-15T21:44:48.507 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T21:44:48.514 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T21:44:48.539 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906220', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:40:12.716921', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDnmgv8AOVyOcHSLuNx7yek9GMfWCx0EyjgIsM38xbzh6WhkY5WqoA0n0FTmFcLOOaeEadFQy6lh35lya6L80f8='} 2024-09-15T21:44:48.539 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T21:44:48.545 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2024-09-15T21:44:48.545 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T21:44:48.585 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f86585b7be0>, signals=[15]) 2024-09-15T21:44:48.585 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T21:44:48.592 INFO:teuthology.task.internal:Opening connections... 2024-09-15T21:44:48.592 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:44:48.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:44:48.669 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T21:44:48.676 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-09-15T21:44:48.748 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-09-15T21:44:48.749 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-09-15T21:44:48.806 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-09-15T21:44:48.806 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-09-15T21:44:48.807 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-09-15T21:44:48.807 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:44:48.808 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-09-15T21:44:48.808 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:44:48.808 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:44:48.808 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-09-15T21:44:48.808 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:44:48.809 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:44:48.809 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:44:48.809 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:44:48.809 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:44:48.809 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:44:48.810 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-09-15T21:44:48.834 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T21:44:48.842 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T21:44:48.846 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T21:44:48.847 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:44:48.864 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T21:44:48.870 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T21:44:48.870 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:44:48.932 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T21:44:48.963 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T21:44:48.977 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T21:44:48.978 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T21:44:48.978 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:44:48.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:44:48.996 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-09-15T21:44:49.054 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-503.el9.x86_64 2024-09-15T21:44:49.054 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.14.0-503.el9.x86_64 2024-09-15T21:44:49.055 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2024-09-15T21:44:52.793 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.2 MB 00:02 2024-09-15T21:44:55.035 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-15T21:45:00.594 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-15T21:45:02.386 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-15T21:45:03.490 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-15T21:45:09.196 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-15T21:45:11.244 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T21:45:11.245 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T21:45:11.332 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:45:11.349 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:45:11.350 INFO:teuthology.orchestra.run.smithi047.stdout:Install 4 Packages 2024-09-15T21:45:11.351 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:45:11.351 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 79 M 2024-09-15T21:45:11.351 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 126 M 2024-09-15T21:45:11.351 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:45:11.756 INFO:teuthology.orchestra.run.smithi047.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 79 kB/s | 23 kB 00:00 2024-09-15T21:45:27.072 INFO:teuthology.orchestra.run.smithi047.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:15 2024-09-15T21:45:39.979 INFO:teuthology.orchestra.run.smithi047.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 1.0 MB/s | 29 MB 00:28 2024-09-15T21:45:47.612 INFO:teuthology.orchestra.run.smithi047.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 978 kB/s | 35 MB 00:36 2024-09-15T21:45:47.613 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:45:47.613 INFO:teuthology.orchestra.run.smithi047.stdout:Total 2.2 MB/s | 79 MB 00:36 2024-09-15T21:45:48.322 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:45:48.709 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:45:48.709 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:45:49.679 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:45:49.679 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:45:52.248 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:45:52.898 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:45:52.981 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:45:55.796 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:45:55.923 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:46:01.049 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:46:01.098 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:46:06.245 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:46:45.140 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:46:45.181 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:46:45.519 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:46:45.520 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:46:45.520 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:46:45.520 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:46:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:46:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:46:46.376 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:46:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T21:46:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T21:46:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T21:46:46.376 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T21:46:46.377 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:46:46.377 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:46:46.699 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:46:46.782 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:46:46.782 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:46:46.782 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T21:46:46.782 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T21:46:46.783 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T21:46:46.783 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T21:46:46.783 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T21:46:46.783 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:46:46.842 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:46:46.842 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:46:46.843 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-09-15T21:46:46.873 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.06 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 82.el9 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Group : Unspecified 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 8274670 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T21:46:46.874 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : builder@centos.org 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:46:46.875 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T21:46:46.876 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T21:46:46.876 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 5.14.0-505.el9.x86_64 2024-09-15T21:46:46.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T21:46:47.508 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-09-15T21:46:48.840 INFO:teuthology.orchestra.run.smithi047.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T21:46:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-09-15T21:46:48.878 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-09-15T21:46:48.894 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.HiKTP69gar 2024-09-15T21:46:48.895 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HiKTP69gar 2024-09-15T21:46:48.961 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.HiKTP69gar 2024-09-15T21:46:49.096 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.HiKTP69gar is 6KB 2024-09-15T21:46:49.147 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.HiKTP69gar 2024-09-15T21:46:49.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T21:46:49.236 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T21:46:49.237 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T21:46:49.237 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T21:46:49.237 INFO:teuthology.orchestra.run.smithi047.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T21:46:49.238 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T21:46:49.390 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-09-15T21:47:19.422 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T21:47:19.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:47:37.917 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-09-15T21:47:46.925 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T21:47:46.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:48:46.989 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T21:48:59.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T21:48:59.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:48:59.366 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-09-15T21:48:59.744 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T21:48:59.744 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-15T21:48:59.744 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-09-15T21:48:59.761 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-505.el9.x86_64 2024-09-15T21:48:59.762 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T21:48:59.762 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T21:48:59.762 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T21:49:00.763 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T21:49:00.763 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T21:49:00.862 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2024-09-15T21:49:00.887 DEBUG:teuthology.parallel:result is None 2024-09-15T21:49:00.887 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T21:49:00.895 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T21:49:00.895 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:49:00.914 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T21:49:00.921 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T21:49:00.931 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T21:49:00.931 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:49:00.986 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T21:49:00.992 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T21:49:00.993 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:49:01.067 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:49:01.078 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:49:01.080 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T21:49:01.087 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T21:49:01.087 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:49:01.115 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T21:49:01.125 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T21:49:01.126 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:49:01.172 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:49:01.260 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:49:01.306 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:49:01.393 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:49:01.435 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:49:01.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:49:01.502 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-09-15T21:49:01.585 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:49:01.843 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T21:49:01.853 INFO:teuthology.task.internal:Starting timer... 2024-09-15T21:49:01.853 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T21:49:01.869 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T21:49:01.882 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-15T21:49:01.883 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-09-15T21:49:01.937 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-09-15T21:49:01.939 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T21:49:01.939 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-09-15T21:49:01.980 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-09-15T21:49:01.980 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive'} 2024-09-15T21:49:01.980 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:49:02.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:49:02.047 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T21:49:02.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:49:02.121 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T21:49:02.133 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T21:49:02.148 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T21:49:02.149 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T21:52:54.532 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com')] 2024-09-15T21:52:54.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T21:52:54.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:52:54.630 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-09-15T21:52:54.709 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-15T21:52:54.709 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T21:52:54.721 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T21:52:54.721 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:52:54.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:52:54.780 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:52:54.795 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:52:54.821 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-09-15T21:52:54.833 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-09-15T21:52:54.848 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:52:54.863 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:52:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:52:54.925 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:52:54.925 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-09-15T21:52:54.925 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:52:54.925 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:52:54.925 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:52:54.926 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:52:54.926 INFO:teuthology.run_tasks:Running task install... 2024-09-15T21:52:54.934 DEBUG:teuthology.task.install:project ceph 2024-09-15T21:52:54.934 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-15T21:52:54.934 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-15T21:52:54.934 INFO:teuthology.task.install:Using flavor: default 2024-09-15T21:52:54.939 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T21:52:54.939 INFO:teuthology.task.install:extra packages: ['librados2'] 2024-09-15T21:52:54.940 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T21:52:54.940 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:52:54.940 INFO:teuthology.packaging:ref: None 2024-09-15T21:52:54.940 INFO:teuthology.packaging:tag: None 2024-09-15T21:52:54.940 INFO:teuthology.packaging:branch: quincy 2024-09-15T21:52:54.940 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:52:54.941 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-15T21:52:55.168 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/ 2024-09-15T21:52:55.169 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1633.g6d025bc2 2024-09-15T21:52:55.281 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/quincy/6d025bc26f3176c910707f8cb4c8d8e076640615/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:52:55.282 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:52:55.282 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:52:55.310 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2024-09-15T21:52:55.311 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:52:55.311 INFO:teuthology.packaging:ref: None 2024-09-15T21:52:55.311 INFO:teuthology.packaging:tag: None 2024-09-15T21:52:55.311 INFO:teuthology.packaging:branch: quincy 2024-09-15T21:52:55.311 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:52:55.312 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:52:55.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:52:55.456 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:52:55.521 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-09-15T21:52:55.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-09-15T21:52:55.758 INFO:teuthology.orchestra.run.smithi047.stdout:45 files removed 2024-09-15T21:52:55.785 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw 2024-09-15T21:52:56.454 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 209 kB/s | 77 kB 00:00 2024-09-15T21:52:56.938 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-09-15T21:52:57.339 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 5.0 kB/s | 1.8 kB 00:00 2024-09-15T21:52:58.814 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.2 MB 00:01 2024-09-15T21:53:00.682 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-15T21:53:05.283 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-15T21:53:06.800 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 19 kB 00:00 2024-09-15T21:53:07.813 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-15T21:53:13.486 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-15T21:53:15.955 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:53:15.970 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:53:15.970 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-15T21:53:15.970 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:53:15.970 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:53:15.970 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 14 M 2024-09-15T21:53:15.970 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-09-15T21:53:15.970 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 3.1 M 2024-09-15T21:53:15.970 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 2.9 M 2024-09-15T21:53:15.970 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 5.0 M 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 22 M 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 25 k 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 637 k 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 457 k 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 4.9 M 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T21:53:15.971 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 45 k 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 120 k 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 156 k 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 320 k 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 295 k 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 99 k 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout:Install 22 Packages 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 2 Packages 2024-09-15T21:53:15.972 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:53:15.973 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 58 M 2024-09-15T21:53:15.973 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:53:16.887 INFO:teuthology.orchestra.run.smithi047.stdout:(1/24): ceph-base-17.2.7-1633.g6d025bc2.el9.x86 10 MB/s | 5.0 MB 00:00 2024-09-15T21:53:16.945 INFO:teuthology.orchestra.run.smithi047.stdout:(2/24): ceph-selinux-17.2.7-1633.g6d025bc2.el9. 431 kB/s | 25 kB 00:00 2024-09-15T21:53:17.029 INFO:teuthology.orchestra.run.smithi047.stdout:(3/24): libcephfs2-17.2.7-1633.g6d025bc2.el9.x8 7.5 MB/s | 637 kB 00:00 2024-09-15T21:53:17.112 INFO:teuthology.orchestra.run.smithi047.stdout:(4/24): libradosstriper1-17.2.7-1633.g6d025bc2. 5.4 MB/s | 457 kB 00:00 2024-09-15T21:53:17.429 INFO:teuthology.orchestra.run.smithi047.stdout:(5/24): ceph-radosgw-17.2.7-1633.g6d025bc2.el9. 14 MB/s | 14 MB 00:01 2024-09-15T21:53:17.522 INFO:teuthology.orchestra.run.smithi047.stdout:(6/24): librgw2-17.2.7-1633.g6d025bc2.el9.x86_6 12 MB/s | 4.9 MB 00:00 2024-09-15T21:53:17.546 INFO:teuthology.orchestra.run.smithi047.stdout:(7/24): python3-ceph-argparse-17.2.7-1633.g6d02 386 kB/s | 45 kB 00:00 2024-09-15T21:53:17.580 INFO:teuthology.orchestra.run.smithi047.stdout:(8/24): python3-ceph-common-17.2.7-1633.g6d025b 2.0 MB/s | 120 kB 00:00 2024-09-15T21:53:17.605 INFO:teuthology.orchestra.run.smithi047.stdout:(9/24): python3-cephfs-17.2.7-1633.g6d025bc2.el 2.6 MB/s | 156 kB 00:00 2024-09-15T21:53:17.630 INFO:teuthology.orchestra.run.smithi047.stdout:(10/24): python3-rados-17.2.7-1633.g6d025bc2.el 6.2 MB/s | 320 kB 00:00 2024-09-15T21:53:17.656 INFO:teuthology.orchestra.run.smithi047.stdout:(11/24): python3-rbd-17.2.7-1633.g6d025bc2.el9. 5.7 MB/s | 295 kB 00:00 2024-09-15T21:53:17.681 INFO:teuthology.orchestra.run.smithi047.stdout:(12/24): python3-rgw-17.2.7-1633.g6d025bc2.el9. 1.9 MB/s | 99 kB 00:00 2024-09-15T21:53:17.900 INFO:teuthology.orchestra.run.smithi047.stdout:(13/24): ceph-common-17.2.7-1633.g6d025bc2.el9. 15 MB/s | 22 MB 00:01 2024-09-15T21:53:17.990 INFO:teuthology.orchestra.run.smithi047.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 146 kB/s | 45 kB 00:00 2024-09-15T21:53:18.007 INFO:teuthology.orchestra.run.smithi047.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 457 kB/s | 160 kB 00:00 2024-09-15T21:53:18.066 INFO:teuthology.orchestra.run.smithi047.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 726 kB/s | 42 kB 00:00 2024-09-15T21:53:18.116 INFO:teuthology.orchestra.run.smithi047.stdout:(17/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-09-15T21:53:18.150 INFO:teuthology.orchestra.run.smithi047.stdout:(18/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-09-15T21:53:18.183 INFO:teuthology.orchestra.run.smithi047.stdout:(19/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2024-09-15T21:53:18.208 INFO:teuthology.orchestra.run.smithi047.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-09-15T21:53:18.250 INFO:teuthology.orchestra.run.smithi047.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 685 kB/s | 67 kB 00:00 2024-09-15T21:53:18.309 INFO:teuthology.orchestra.run.smithi047.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-15T21:53:18.526 INFO:teuthology.orchestra.run.smithi047.stdout:(23/24): librbd1-17.2.7-1633.g6d025bc2.el9.x86_ 11 MB/s | 2.9 MB 00:00 2024-09-15T21:53:18.584 INFO:teuthology.orchestra.run.smithi047.stdout:(24/24): librados2-17.2.7-1633.g6d025bc2.el9.x8 8.2 MB/s | 3.1 MB 00:00 2024-09-15T21:53:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:53:18.590 INFO:teuthology.orchestra.run.smithi047.stdout:Total 22 MB/s | 58 MB 00:02 2024-09-15T21:53:18.658 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:53:18.696 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:53:18.696 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:53:19.373 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:53:19.373 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:53:20.648 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:53:21.040 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-15T21:53:21.120 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/26 2024-09-15T21:53:21.221 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/26 2024-09-15T21:53:21.310 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/26 2024-09-15T21:53:21.338 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/26 2024-09-15T21:53:21.427 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/26 2024-09-15T21:53:21.484 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-15T21:53:21.518 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-15T21:53:21.982 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-15T21:53:22.008 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 8/26 2024-09-15T21:53:22.071 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 8/26 2024-09-15T21:53:22.108 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-15T21:53:22.151 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9. 10/26 2024-09-15T21:53:22.187 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 11/26 2024-09-15T21:53:22.252 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 12/26 2024-09-15T21:53:22.277 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 13/26 2024-09-15T21:53:22.343 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 13/26 2024-09-15T21:53:22.383 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-15T21:53:22.428 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-15T21:53:22.469 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-15T21:53:22.788 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-15T21:53:22.823 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 18/26 2024-09-15T21:53:22.914 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 18/26 2024-09-15T21:53:22.990 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 19/26 2024-09-15T21:53:23.025 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x8 20/26 2024-09-15T21:53:25.560 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 21/26 2024-09-15T21:53:25.588 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 21/26 2024-09-15T21:53:26.135 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 21/26 2024-09-15T21:53:26.167 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 22/26 2024-09-15T21:53:26.270 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 22/26 2024-09-15T21:53:26.271 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T21:53:26.271 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T21:53:26.271 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:53:26.310 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 23/26 2024-09-15T21:53:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 23/26 2024-09-15T21:53:36.710 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-09-15T21:53:36.710 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-09-15T21:53:36.710 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-09-15T21:53:36.710 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-09-15T21:53:36.710 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-09-15T21:53:36.710 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-09-15T21:53:36.710 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-09-15T21:53:36.710 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:53:38.077 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 24/26 2024-09-15T21:53:38.157 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 24/26 2024-09-15T21:53:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:53:38.157 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T21:53:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:53:38.158 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:53:38.158 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:53:38.159 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T21:53:38.246 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-15T21:53:38.247 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T21:53:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T21:53:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/26 2024-09-15T21:53:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/26 2024-09-15T21:53:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/26 2024-09-15T21:53:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/26 2024-09-15T21:53:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 5/26 2024-09-15T21:53:39.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 6/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 7/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9. 8/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x8 9/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 10/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 11/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 12/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 13/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-15T21:53:39.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-15T21:53:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-15T21:53:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-15T21:53:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-15T21:53:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-15T21:53:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 23/26 2024-09-15T21:53:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-15T21:53:39.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 25/26 2024-09-15T21:53:40.534 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-15T21:53:40.534 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:53:40.534 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-09-15T21:53:40.534 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.534 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.534 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:53:40.534 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.534 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.534 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.534 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.535 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.536 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T21:53:40.536 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.536 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.536 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:53:40.536 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T21:53:40.536 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:53:40.536 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:53:40.912 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-test 2024-09-15T21:53:41.510 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:28 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:53:41.648 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:53:41.648 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:53:41.649 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-15T21:53:41.649 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:53:41.649 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:53:41.649 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 48 M 2024-09-15T21:53:41.649 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-15T21:53:41.649 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 163 k 2024-09-15T21:53:41.649 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T21:53:41.649 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T21:53:41.649 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:53:41.649 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:53:41.650 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:53:41.650 INFO:teuthology.orchestra.run.smithi047.stdout:Install 4 Packages 2024-09-15T21:53:41.650 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:53:41.650 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 48 M 2024-09-15T21:53:41.650 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 193 M 2024-09-15T21:53:41.650 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:53:41.930 INFO:teuthology.orchestra.run.smithi047.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-15T21:53:41.972 INFO:teuthology.orchestra.run.smithi047.stdout:(2/4): libcephsqlite-17.2.7-1633.g6d025bc2.el9. 593 kB/s | 163 kB 00:00 2024-09-15T21:53:41.997 INFO:teuthology.orchestra.run.smithi047.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 952 kB/s | 64 kB 00:00 2024-09-15T21:53:52.355 INFO:teuthology.orchestra.run.smithi047.stdout:(4/4): ceph-test-17.2.7-1633.g6d025bc2.el9.x86_ 4.5 MB/s | 48 MB 00:10 2024-09-15T21:53:52.357 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:53:52.357 INFO:teuthology.orchestra.run.smithi047.stdout:Total 4.5 MB/s | 48 MB 00:10 2024-09-15T21:53:52.385 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:53:52.406 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:53:52.406 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:53:52.791 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:53:52.792 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:53:53.340 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:53:53.431 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T21:53:53.480 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T21:53:53.506 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/4 2024-09-15T21:53:58.155 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/4 2024-09-15T21:53:58.241 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/4 2024-09-15T21:54:00.354 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/4 2024-09-15T21:54:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/4 2024-09-15T21:54:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/4 2024-09-15T21:54:00.355 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T21:54:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T21:54:00.694 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:00.694 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:54:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T21:54:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T21:54:00.695 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:00.695 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:00.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph 2024-09-15T21:54:01.561 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:00:48 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:01.703 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:01.705 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:01.705 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-15T21:54:01.705 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:01.705 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:54:01.705 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 6.5 k 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 2.1 M 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 1.4 M 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1633.g6d025bc2.el9 ceph-noarch 240 k 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 3.9 M 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 15 M 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T21:54:01.706 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T21:54:01.707 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:01.708 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:54:01.709 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:01.709 INFO:teuthology.orchestra.run.smithi047.stdout:Install 34 Packages 2024-09-15T21:54:01.709 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:01.709 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 27 M 2024-09-15T21:54:01.709 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 96 M 2024-09-15T21:54:01.709 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:54:02.297 INFO:teuthology.orchestra.run.smithi047.stdout:(1/34): ceph-17.2.7-1633.g6d025bc2.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-15T21:54:02.489 INFO:teuthology.orchestra.run.smithi047.stdout:(2/34): ceph-mds-17.2.7-1633.g6d025bc2.el9.x86_ 6.5 MB/s | 2.1 MB 00:00 2024-09-15T21:54:02.531 INFO:teuthology.orchestra.run.smithi047.stdout:(3/34): ceph-mgr-17.2.7-1633.g6d025bc2.el9.x86_ 3.9 MB/s | 1.4 MB 00:00 2024-09-15T21:54:02.615 INFO:teuthology.orchestra.run.smithi047.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1633.g6d02 2.8 MB/s | 240 kB 00:00 2024-09-15T21:54:02.699 INFO:teuthology.orchestra.run.smithi047.stdout:(5/34): ceph-mon-17.2.7-1633.g6d025bc2.el9.x86_ 9.8 MB/s | 3.9 MB 00:00 2024-09-15T21:54:02.991 INFO:teuthology.orchestra.run.smithi047.stdout:(6/34): ceph-osd-17.2.7-1633.g6d025bc2.el9.x86_ 31 MB/s | 15 MB 00:00 2024-09-15T21:54:03.015 INFO:teuthology.orchestra.run.smithi047.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 632 kB/s | 253 kB 00:00 2024-09-15T21:54:03.066 INFO:teuthology.orchestra.run.smithi047.stdout:(8/34): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-09-15T21:54:03.124 INFO:teuthology.orchestra.run.smithi047.stdout:(9/34): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-09-15T21:54:03.158 INFO:teuthology.orchestra.run.smithi047.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 2.7 MB/s | 1.2 MB 00:00 2024-09-15T21:54:03.184 INFO:teuthology.orchestra.run.smithi047.stdout:(11/34): python3-ply-3.11-14.el9.noarch.rpm 553 kB/s | 106 kB 00:00 2024-09-15T21:54:03.209 INFO:teuthology.orchestra.run.smithi047.stdout:(12/34): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-15T21:54:03.284 INFO:teuthology.orchestra.run.smithi047.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 347 kB/s | 35 kB 00:00 2024-09-15T21:54:03.334 INFO:teuthology.orchestra.run.smithi047.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 616 kB/s | 77 kB 00:00 2024-09-15T21:54:03.359 INFO:teuthology.orchestra.run.smithi047.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 679 kB/s | 51 kB 00:00 2024-09-15T21:54:03.385 INFO:teuthology.orchestra.run.smithi047.stdout:(16/34): python3-mako-1.1.4-6.el9.noarch.rpm 760 kB/s | 172 kB 00:00 2024-09-15T21:54:03.410 INFO:teuthology.orchestra.run.smithi047.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 549 kB/s | 42 kB 00:00 2024-09-15T21:54:03.461 INFO:teuthology.orchestra.run.smithi047.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 431 kB/s | 43 kB 00:00 2024-09-15T21:54:03.486 INFO:teuthology.orchestra.run.smithi047.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 468 kB/s | 11 kB 00:00 2024-09-15T21:54:03.511 INFO:teuthology.orchestra.run.smithi047.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 711 kB/s | 18 kB 00:00 2024-09-15T21:54:03.545 INFO:teuthology.orchestra.run.smithi047.stdout:(21/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-15T21:54:03.570 INFO:teuthology.orchestra.run.smithi047.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 395 kB/s | 23 kB 00:00 2024-09-15T21:54:03.595 INFO:teuthology.orchestra.run.smithi047.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-15T21:54:03.621 INFO:teuthology.orchestra.run.smithi047.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-15T21:54:03.646 INFO:teuthology.orchestra.run.smithi047.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-15T21:54:03.671 INFO:teuthology.orchestra.run.smithi047.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-15T21:54:03.700 INFO:teuthology.orchestra.run.smithi047.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-15T21:54:03.733 INFO:teuthology.orchestra.run.smithi047.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 266 kB/s | 16 kB 00:00 2024-09-15T21:54:03.759 INFO:teuthology.orchestra.run.smithi047.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-15T21:54:03.784 INFO:teuthology.orchestra.run.smithi047.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-15T21:54:03.809 INFO:teuthology.orchestra.run.smithi047.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-15T21:54:03.835 INFO:teuthology.orchestra.run.smithi047.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 799 kB/s | 20 kB 00:00 2024-09-15T21:54:03.860 INFO:teuthology.orchestra.run.smithi047.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-15T21:54:03.893 INFO:teuthology.orchestra.run.smithi047.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-15T21:54:03.898 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:54:03.898 INFO:teuthology.orchestra.run.smithi047.stdout:Total 12 MB/s | 27 MB 00:02 2024-09-15T21:54:04.037 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:54:04.077 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:54:04.078 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:54:04.496 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:54:04.496 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:54:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:54:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-15T21:54:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-15T21:54:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-15T21:54:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-15T21:54:06.077 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-15T21:54:06.124 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-15T21:54:06.159 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-15T21:54:06.188 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-15T21:54:06.223 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-15T21:54:06.256 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-15T21:54:06.298 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-15T21:54:06.349 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-15T21:54:06.391 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-15T21:54:06.441 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-15T21:54:06.481 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T21:54:06.558 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-15T21:54:06.709 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-15T21:54:06.792 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-15T21:54:06.843 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-15T21:54:06.889 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-15T21:54:06.948 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-15T21:54:07.012 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-15T21:54:07.226 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-15T21:54:07.279 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-15T21:54:07.368 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-15T21:54:07.467 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-15T21:54:07.515 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-15T21:54:07.643 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-15T21:54:07.807 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9. 29/34 2024-09-15T21:54:07.838 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 30/34 2024-09-15T21:54:07.865 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 30/34 2024-09-15T21:54:07.866 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:54:07.866 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T21:54:07.866 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:54:07.866 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:54:07.866 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:09.248 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 31/34 2024-09-15T21:54:09.277 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 31/34 2024-09-15T21:54:09.277 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:54:09.277 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T21:54:09.277 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:54:09.277 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:54:09.277 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:09.673 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 32/34 2024-09-15T21:54:09.699 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 32/34 2024-09-15T21:54:09.699 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:54:09.699 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T21:54:09.699 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:54:09.699 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:54:09.699 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:09.915 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 33/34 2024-09-15T21:54:09.941 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 33/34 2024-09-15T21:54:09.941 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:54:09.941 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T21:54:09.941 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:54:09.941 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:54:09.941 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:10.007 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 34/34 2024-09-15T21:54:11.714 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 34/34 2024-09-15T21:54:11.714 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/34 2024-09-15T21:54:11.714 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/34 2024-09-15T21:54:11.714 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 3/34 2024-09-15T21:54:11.714 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 4/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 5/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9. 6/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-15T21:54:11.715 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-15T21:54:11.716 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-15T21:54:11.717 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-15T21:54:11.717 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-15T21:54:11.718 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-15T21:54:11.718 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-15T21:54:11.718 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-15T21:54:11.718 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-15T21:54:12.802 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9.noarch 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T21:54:12.803 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T21:54:12.804 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T21:54:12.805 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T21:54:12.806 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T21:54:12.806 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:12.806 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:13.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-base 2024-09-15T21:54:13.778 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:00 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:13.871 INFO:teuthology.orchestra.run.smithi047.stdout:Package ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-15T21:54:13.915 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:13.916 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-15T21:54:13.917 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:13.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T21:54:14.565 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:01 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:14.704 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 137 k 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:14.705 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 137 k 2024-09-15T21:54:14.706 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 379 k 2024-09-15T21:54:14.706 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:54:14.898 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-immutable-object-cache-17.2.7-1633.g6d025b 715 kB/s | 137 kB 00:00 2024-09-15T21:54:14.898 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:54:14.899 INFO:teuthology.orchestra.run.smithi047.stdout:Total 709 kB/s | 137 kB 00:00 2024-09-15T21:54:14.899 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:54:14.907 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:54:14.907 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:54:14.967 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:54:14.968 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:54:15.121 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:54:15.191 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2. 1/1 2024-09-15T21:54:15.217 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2. 1/1 2024-09-15T21:54:15.217 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:54:15.217 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T21:54:15.217 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:16.209 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2. 1/1 2024-09-15T21:54:16.209 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:16.209 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:54:16.209 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:16.209 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:16.209 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:16.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-mgr 2024-09-15T21:54:17.021 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:04 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:17.115 INFO:teuthology.orchestra.run.smithi047.stdout:Package ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-15T21:54:17.159 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:17.160 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-15T21:54:17.161 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:17.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-fuse 2024-09-15T21:54:17.803 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:04 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:17.942 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:17.942 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:17.942 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-15T21:54:17.943 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:17.943 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:54:17.943 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 814 k 2024-09-15T21:54:17.943 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-15T21:54:17.943 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T21:54:17.943 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:17.943 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:54:17.943 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:17.943 INFO:teuthology.orchestra.run.smithi047.stdout:Install 2 Packages 2024-09-15T21:54:17.943 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:17.944 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 893 k 2024-09-15T21:54:17.944 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 2.6 M 2024-09-15T21:54:17.944 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:54:18.127 INFO:teuthology.orchestra.run.smithi047.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 502 kB/s | 80 kB 00:00 2024-09-15T21:54:18.269 INFO:teuthology.orchestra.run.smithi047.stdout:(2/2): ceph-fuse-17.2.7-1633.g6d025bc2.el9.x86_ 2.6 MB/s | 814 kB 00:00 2024-09-15T21:54:18.269 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:54:18.270 INFO:teuthology.orchestra.run.smithi047.stdout:Total 2.7 MB/s | 893 kB 00:00 2024-09-15T21:54:18.282 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:54:18.306 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:54:18.306 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:54:18.383 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:54:18.383 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:54:18.628 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:54:18.789 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T21:54:18.865 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/2 2024-09-15T21:54:19.565 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2/2 2024-09-15T21:54:19.565 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/2 2024-09-15T21:54:19.871 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T21:54:19.871 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:19.871 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:54:19.871 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T21:54:19.872 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:19.872 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:20.138 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librados-devel 2024-09-15T21:54:20.725 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:07 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:20.861 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:20.862 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:20.862 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-09-15T21:54:20.862 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:20.862 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:54:20.862 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 126 k 2024-09-15T21:54:20.862 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:20.862 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:54:20.862 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:20.862 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-15T21:54:20.863 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:20.863 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 126 k 2024-09-15T21:54:20.863 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 448 k 2024-09-15T21:54:20.863 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:54:21.074 INFO:teuthology.orchestra.run.smithi047.stdout:librados-devel-17.2.7-1633.g6d025bc2.el9.x86_64 597 kB/s | 126 kB 00:00 2024-09-15T21:54:21.075 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:54:21.075 INFO:teuthology.orchestra.run.smithi047.stdout:Total 593 kB/s | 126 kB 00:00 2024-09-15T21:54:21.075 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:54:21.081 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:54:21.081 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:54:21.141 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:54:21.141 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:54:21.288 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:54:21.389 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:21.769 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:22.016 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:22.017 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:22.017 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:54:22.017 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:22.017 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:22.017 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:22.243 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install libcephfs2 2024-09-15T21:54:22.831 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:09 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:22.925 INFO:teuthology.orchestra.run.smithi047.stdout:Package libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-15T21:54:22.970 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:22.971 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-15T21:54:22.971 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:23.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install libcephfs-devel 2024-09-15T21:54:23.616 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:10 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:23.751 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:23.752 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:23.752 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-09-15T21:54:23.752 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:23.753 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:54:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 25 k 2024-09-15T21:54:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:23.753 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:54:23.753 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:23.753 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-15T21:54:23.753 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:23.753 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 25 k 2024-09-15T21:54:23.753 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 110 k 2024-09-15T21:54:23.753 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:54:23.903 INFO:teuthology.orchestra.run.smithi047.stdout:libcephfs-devel-17.2.7-1633.g6d025bc2.el9.x86_6 167 kB/s | 25 kB 00:00 2024-09-15T21:54:23.904 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:54:23.904 INFO:teuthology.orchestra.run.smithi047.stdout:Total 165 kB/s | 25 kB 00:00 2024-09-15T21:54:23.904 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:54:23.907 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:54:23.907 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:54:23.909 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:54:23.909 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:54:23.994 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:54:24.100 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:24.487 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:24.726 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:24.726 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:24.726 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:54:24.726 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:24.726 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:24.726 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:24.888 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librados2 2024-09-15T21:54:25.485 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:12 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:25.582 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-15T21:54:25.627 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:25.629 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-15T21:54:25.629 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:25.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librbd1 2024-09-15T21:54:26.287 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:13 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:26.383 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-15T21:54:26.427 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:26.428 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-15T21:54:26.429 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:26.491 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rados 2024-09-15T21:54:27.075 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:14 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:27.170 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-15T21:54:27.214 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:27.215 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-15T21:54:27.215 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:27.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rgw 2024-09-15T21:54:27.855 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:14 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:27.949 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-15T21:54:27.993 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:27.994 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-15T21:54:27.994 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:28.067 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-cephfs 2024-09-15T21:54:28.644 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:15 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:28.736 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-15T21:54:28.780 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:28.781 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-15T21:54:28.782 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:28.847 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install python3-rbd 2024-09-15T21:54:29.427 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:16 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:29.521 INFO:teuthology.orchestra.run.smithi047.stdout:Package python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-15T21:54:29.566 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:29.567 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-15T21:54:29.567 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:29.631 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-fuse 2024-09-15T21:54:30.219 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:17 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:30.359 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:30.360 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:30.360 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 82 k 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 82 k 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 222 k 2024-09-15T21:54:30.361 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:54:30.549 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-fuse-17.2.7-1633.g6d025bc2.el9.x86_64.rpm 436 kB/s | 82 kB 00:00 2024-09-15T21:54:30.549 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:54:30.550 INFO:teuthology.orchestra.run.smithi047.stdout:Total 433 kB/s | 82 kB 00:00 2024-09-15T21:54:30.550 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:54:30.557 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:54:30.557 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:54:30.617 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:54:30.617 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:54:30.756 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:54:30.857 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:31.261 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:31.535 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:31.535 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:31.535 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:54:31.535 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:31.535 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:31.535 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:31.769 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-mirror 2024-09-15T21:54:32.352 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:19 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:32.489 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:32.489 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:32.489 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repo Size 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 2.9 M 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 2.9 M 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 12 M 2024-09-15T21:54:32.490 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:54:32.819 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-mirror-17.2.7-1633.g6d025bc2.el9.x86_64.rpm 9.0 MB/s | 2.9 MB 00:00 2024-09-15T21:54:32.819 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:54:32.820 INFO:teuthology.orchestra.run.smithi047.stdout:Total 8.9 MB/s | 2.9 MB 00:00 2024-09-15T21:54:32.820 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:54:32.830 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:54:32.830 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:54:32.908 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:54:32.908 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:54:33.332 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:54:33.411 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:33.438 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:54:33.438 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T21:54:33.438 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:54:33.438 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:54:33.438 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:34.490 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:34.490 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:34.490 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:54:34.490 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:34.490 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:34.490 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:34.722 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install rbd-nbd 2024-09-15T21:54:35.315 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:22 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:35.452 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:35.453 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:35.453 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-09-15T21:54:35.453 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:35.453 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-15T21:54:35.453 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:17.2.7-1633.g6d025bc2.el9 ceph 165 k 2024-09-15T21:54:35.453 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:35.454 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:54:35.454 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:54:35.454 INFO:teuthology.orchestra.run.smithi047.stdout:Install 1 Package 2024-09-15T21:54:35.454 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:35.454 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 165 k 2024-09-15T21:54:35.454 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 477 k 2024-09-15T21:54:35.454 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:54:35.664 INFO:teuthology.orchestra.run.smithi047.stdout:rbd-nbd-17.2.7-1633.g6d025bc2.el9.x86_64.rpm 783 kB/s | 165 kB 00:00 2024-09-15T21:54:35.665 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:54:35.665 INFO:teuthology.orchestra.run.smithi047.stdout:Total 778 kB/s | 165 kB 00:00 2024-09-15T21:54:35.665 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:54:35.673 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:54:35.674 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:54:35.733 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:54:35.734 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:54:35.890 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:54:35.997 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:36.413 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:36.689 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 1/1 2024-09-15T21:54:36.689 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:36.689 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:54:36.689 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 2024-09-15T21:54:36.689 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:36.689 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:36.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install librados2 2024-09-15T21:54:37.517 INFO:teuthology.orchestra.run.smithi047.stdout:Last metadata expiration check: 0:01:24 ago on Sun 15 Sep 2024 09:53:13 PM UTC. 2024-09-15T21:54:37.612 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 is already installed. 2024-09-15T21:54:37.656 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:54:37.657 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-09-15T21:54:37.657 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:54:37.725 DEBUG:teuthology.parallel:result is None 2024-09-15T21:54:37.726 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:54:37.726 INFO:teuthology.packaging:ref: None 2024-09-15T21:54:37.726 INFO:teuthology.packaging:tag: None 2024-09-15T21:54:37.726 INFO:teuthology.packaging:branch: quincy 2024-09-15T21:54:37.726 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:54:37.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-15T21:54:37.952 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:54:37.974 INFO:teuthology.orchestra.run.smithi047.stdout:17.2.7-1633.g6d025bc2.el9 2024-09-15T21:54:37.975 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1633.g6d025bc2.el9 2024-09-15T21:54:37.975 INFO:teuthology.task.install:The correct ceph version 17.2.7-1633.g6d025bc2 is installed. 2024-09-15T21:54:37.976 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T21:54:37.977 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:37.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:54:38.041 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T21:54:38.041 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:38.041 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:54:38.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:54:38.173 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T21:54:38.173 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:38.173 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:54:38.238 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:54:38.304 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T21:54:38.304 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:38.304 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:54:38.369 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:54:38.435 INFO:teuthology.run_tasks:Running task print... 2024-09-15T21:54:38.445 INFO:teuthology.task.print:**** done installing quincy 2024-09-15T21:54:38.446 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T21:54:38.536 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T21:54:38.536 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2024-09-15T21:54:38.562 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T21:54:38.562 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T21:54:38.628 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T21:54:38.628 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T21:54:38.707 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T21:54:38.708 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T21:54:38.708 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906220', 'branch': 'squid', 'description': 'fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7906220', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'install ceph/quincy latest\n'}], 'name': 'teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 3393, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'sleep_before_teardown': 0, 'subset': '294/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4', 'targets': {'smithi047.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDnmgv8AOVyOcHSLuNx7yek9GMfWCx0EyjgIsM38xbzh6WhkY5WqoA0n0FTmFcLOOaeEadFQy6lh35lya6L80f8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'quincy', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, {'print': '**** done installing quincy'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client quincy']}}, {'print': '**** done ceph'}, {'print': '*** upgrading, no cephfs present'}, {'exec': {'mon.a': ['ceph fs dump']}}, {'install.upgrade': {'mon.a': None}}, {'print': '**** done install.upgrade'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph versions', 'ceph osd dump -f json-pretty', 'ceph fs dump', 'ceph osd require-osd-release squid', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_21:24:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T21:54:38.709 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T21:54:38.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T21:54:38.819 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:38.820 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T21:54:38.874 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T21:54:38.874 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-09-15T21:54:38.930 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T21:54:38.930 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:54:38.931 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T21:54:38.931 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:54:38.931 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:54:38.931 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-15 21:54:33.694845074 +0000 2024-09-15T21:54:38.931 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-15 21:52:34.311179195 +0000 2024-09-15T21:54:38.931 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-15 21:52:34.311179195 +0000 2024-09-15T21:54:38.931 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-15 21:52:34.311179195 +0000 2024-09-15T21:54:38.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T21:54:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-15T21:54:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-15T21:54:38.994 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000101967 s, 5.0 MB/s 2024-09-15T21:54:38.995 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T21:54:39.052 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-09-15T21:54:39.108 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T21:54:39.108 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:54:39.108 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-15T21:54:39.108 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:54:39.108 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:54:39.109 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-15 21:54:33.694845074 +0000 2024-09-15T21:54:39.109 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-15 21:52:34.637172411 +0000 2024-09-15T21:54:39.109 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-15 21:52:34.637172411 +0000 2024-09-15T21:54:39.109 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-15 21:52:34.637172411 +0000 2024-09-15T21:54:39.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T21:54:39.172 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-15T21:54:39.172 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-15T21:54:39.172 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 8.3913e-05 s, 6.1 MB/s 2024-09-15T21:54:39.173 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T21:54:39.230 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-09-15T21:54:39.287 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T21:54:39.287 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:54:39.287 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-15T21:54:39.287 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:54:39.287 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:54:39.287 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-15 21:54:33.694845074 +0000 2024-09-15T21:54:39.288 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-15 21:52:34.968165520 +0000 2024-09-15T21:54:39.288 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-15 21:52:34.968165520 +0000 2024-09-15T21:54:39.288 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-15 21:52:34.968165520 +0000 2024-09-15T21:54:39.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T21:54:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-15T21:54:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-15T21:54:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.7393e-05 s, 5.3 MB/s 2024-09-15T21:54:39.353 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T21:54:39.410 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-09-15T21:54:39.466 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T21:54:39.466 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:54:39.466 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-15T21:54:39.466 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:54:39.467 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:54:39.467 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-15 21:54:33.695845054 +0000 2024-09-15T21:54:39.467 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-15 21:52:35.307158462 +0000 2024-09-15T21:54:39.467 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-15 21:52:35.307158462 +0000 2024-09-15T21:54:39.467 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-15 21:52:35.307158462 +0000 2024-09-15T21:54:39.467 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T21:54:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-15T21:54:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-15T21:54:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000101579 s, 5.0 MB/s 2024-09-15T21:54:39.532 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T21:54:39.589 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:54:39.589 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-09-15T21:54:39.589 INFO:tasks.ceph:Generating config... 2024-09-15T21:54:39.590 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-15T21:54:39.590 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-15T21:54:39.590 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2024-09-15T21:54:39.590 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2024-09-15T21:54:39.590 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2024-09-15T21:54:39.591 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-15T21:54:39.591 INFO:tasks.ceph:[client] debug client = 20 2024-09-15T21:54:39.591 INFO:tasks.ceph:[client] debug ms = 1 2024-09-15T21:54:39.591 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-15T21:54:39.591 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-15T21:54:39.591 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-15T21:54:39.591 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-15T21:54:39.591 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-15T21:54:39.591 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-15T21:54:39.591 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-15T21:54:39.591 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-15T21:54:39.591 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-15T21:54:39.591 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-15T21:54:39.591 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2024-09-15T21:54:39.592 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-15T21:54:39.592 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-15T21:54:39.592 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T21:54:39.593 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-15T21:54:39.594 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T21:54:39.594 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T21:54:39.678 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T21:54:39.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T21:54:39.722 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:54:39.789 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.47'), ('mon.b', '[v2:172.21.15.47:3301,v1:172.21.15.47:6790]'), ('mon.c', '[v2:172.21.15.47:3302,v1:172.21.15.47:6791]')] 2024-09-15T21:54:39.790 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.47,[v2:172.21.15.47:3301,v1:172.21.15.47:6790],[v2:172.21.15.47:3302,v1:172.21.15.47:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-15T21:54:39.791 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:39.791 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:54:39.846 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.47 --addv b '[v2:172.21.15.47:3301,v1:172.21.15.47:6790]' --addv c '[v2:172.21.15.47:3302,v1:172.21.15.47:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:54:39.929 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:54:39.929 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: generated fsid 519e4c2c-d517-4539-a86b-3d427c67bae0 2024-09-15T21:54:39.929 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = octopus 2024-09-15T21:54:39.929 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 0 2024-09-15T21:54:39.929 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 519e4c2c-d517-4539-a86b-3d427c67bae0 2024-09-15T21:54:39.929 INFO:teuthology.orchestra.run.smithi047.stdout:last_changed 2024-09-15T21:54:39.928551+0000 2024-09-15T21:54:39.929 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-09-15T21:54:39.928551+0000 2024-09-15T21:54:39.930 INFO:teuthology.orchestra.run.smithi047.stdout:min_mon_release 15 (octopus) 2024-09-15T21:54:39.930 INFO:teuthology.orchestra.run.smithi047.stdout:election_strategy: 1 2024-09-15T21:54:39.930 INFO:teuthology.orchestra.run.smithi047.stdout:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-09-15T21:54:39.930 INFO:teuthology.orchestra.run.smithi047.stdout:1: [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] mon.b 2024-09-15T21:54:39.930 INFO:teuthology.orchestra.run.smithi047.stdout:2: [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] mon.c 2024-09-15T21:54:39.930 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T21:54:39.931 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:54:39.987 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 519e4c2c-d517-4539-a86b-3d427c67bae0... 2024-09-15T21:54:39.988 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T21:54:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-09-15T21:54:40.071 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-09-15T21:54:40.072 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-09-15T21:54:40.073 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-09-15T21:54:40.074 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,[v2:172.21.15.47:3301,v1:172.21.15.47:6790],[v2:172.21.15.47:3302,v1:172.21.15.47:6791]" 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on pool no app = False 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: ms bind msgr2 = False 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore warn on legacy statfs = False 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore warn on no per pool omap = False 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn min per osd = 0 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 519e4c2c-d517-4539-a86b-3d427c67bae0 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-09-15T21:54:40.075 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug trim objects = true 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.076 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-09-15T21:54:40.077 INFO:teuthology.orchestra.run.smithi047.stdout: bdev async discard = True 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: bdev enable discard = True 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore allocator = bitmap 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 1/20 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 1/20 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 4/10 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-09-15T21:54:40.078 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-09-15T21:54:40.079 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T21:54:40.080 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: mon down mkfs grace = 300 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = False 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on pool no app = False 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-09-15T21:54:40.081 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2024-09-15T21:54:40.082 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2024-09-15T21:54:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds balancer = 20 2024-09-15T21:54:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-09-15T21:54:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2024-09-15T21:54:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2024-09-15T21:54:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2024-09-15T21:54:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2024-09-15T21:54:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-09-15T21:54:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-09-15T21:54:40.083 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-09-15T21:54:40.084 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T21:54:40.084 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T21:54:40.135 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T21:54:40.135 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:40.135 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T21:54:40.151 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:40.151 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T21:54:40.207 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:54:40.207 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:40.207 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T21:54:40.207 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:54:40.282 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:40.283 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:54:40.338 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T21:54:40.338 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T21:54:40.338 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:54:40.427 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:54:40.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T21:54:40.477 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T21:54:40.480 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T21:54:40.480 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T21:54:40.480 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T21:54:40.480 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-09-15T21:54:40.480 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T21:54:40.506 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:54:40.506 INFO:tasks.ceph:role: osd.0 2024-09-15T21:54:40.506 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:54:40.506 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T21:54:40.572 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:54:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:54:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:54:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:54:40.572 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:54:40.572 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:54:40.572 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:54:40.572 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:54:40.573 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:54:40.573 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:54:40.608 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-09-15T21:54:40.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-09-15T21:54:40.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T21:54:40.681 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T21:54:40.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T21:54:40.818 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:54:40.818 INFO:tasks.ceph:role: osd.1 2024-09-15T21:54:40.818 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:54:40.819 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T21:54:40.883 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:54:40.884 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:54:40.884 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:54:40.884 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:54:40.884 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:54:40.884 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:54:40.884 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:54:40.884 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:54:40.884 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:54:40.884 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:54:40.919 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-09-15T21:54:40.921 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-09-15T21:54:40.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T21:54:40.991 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T21:54:41.062 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T21:54:41.128 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:54:41.128 INFO:tasks.ceph:role: osd.2 2024-09-15T21:54:41.128 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:54:41.129 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T21:54:41.194 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:54:41.194 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:54:41.194 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:54:41.194 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:54:41.194 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:54:41.194 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:54:41.194 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:54:41.195 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:54:41.195 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:54:41.195 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:54:41.231 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-09-15T21:54:41.233 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-09-15T21:54:41.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T21:54:41.304 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T21:54:41.375 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T21:54:41.441 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:54:41.441 INFO:tasks.ceph:role: osd.3 2024-09-15T21:54:41.441 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:54:41.441 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T21:54:41.506 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:54:41.507 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:54:41.507 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:54:41.507 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:54:41.507 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:54:41.507 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:54:41.507 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:54:41.507 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:54:41.507 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:54:41.507 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:54:41.542 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-09-15T21:54:41.544 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-09-15T21:54:41.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-15T21:54:41.613 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T21:54:41.684 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:54:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:41.779+0000 7ff8b8723440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T21:54:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:41.779+0000 7ff8b8723440 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T21:54:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:41.779+0000 7ff8b8723440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T21:54:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:41.779+0000 7ff8b8723440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T21:54:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:41.779+0000 7ff8b8723440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T21:54:41.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:41.779+0000 7ff8b8723440 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T21:54:42.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:42.306+0000 7ff8b8723440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:43.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:43.076+0000 7ff8b8723440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:43.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:43.839+0000 7ff8b8723440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:44.642 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T21:54:44.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:54:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:44.763+0000 7f212c618440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T21:54:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:44.763+0000 7f212c618440 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T21:54:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:44.763+0000 7f212c618440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T21:54:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:44.763+0000 7f212c618440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T21:54:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:44.763+0000 7f212c618440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T21:54:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:44.764+0000 7f212c618440 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T21:54:45.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:45.288+0000 7f212c618440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:46.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:46.050+0000 7f212c618440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:46.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:46.812+0000 7f212c618440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:47.615 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T21:54:47.645 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:54:47.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:47.730+0000 7fdbd8012440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T21:54:47.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:47.730+0000 7fdbd8012440 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T21:54:47.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:47.730+0000 7fdbd8012440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T21:54:47.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:47.730+0000 7fdbd8012440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T21:54:47.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:47.730+0000 7fdbd8012440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T21:54:47.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:47.731+0000 7fdbd8012440 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T21:54:48.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:48.254+0000 7fdbd8012440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:49.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:49.016+0000 7fdbd8012440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:49.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:49.778+0000 7fdbd8012440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:50.573 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T21:54:50.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:54:50.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:50.685+0000 7f63c3124440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T21:54:50.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:50.685+0000 7f63c3124440 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T21:54:50.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:50.685+0000 7f63c3124440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T21:54:50.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:50.685+0000 7f63c3124440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T21:54:50.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:50.686+0000 7f63c3124440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T21:54:50.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:50.686+0000 7f63c3124440 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T21:54:51.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:51.210+0000 7f63c3124440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:51.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:51.973+0000 7f63c3124440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:52.736+0000 7f63c3124440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T21:54:53.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T21:54:53.568 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T21:54:53.568 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:53.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T21:54:53.638 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:53.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-15T21:54:53.707 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:53.708 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T21:54:53.777 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:53.777 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T21:54:53.845 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:53.845 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T21:54:53.912 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:54:53.912 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T21:54:53.980 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T21:54:53.981 DEBUG:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T21:54:54.045 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2024-09-15T21:54:54.045 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCgV+dmvrZqGRAAUSViMZLmcqoM9m74FKIFJA== 2024-09-15T21:54:54.045 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.y] 2024-09-15T21:54:54.045 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCgV+dm5C1yHBAANPqcpbSBAmwG1u1F/uWmRQ== 2024-09-15T21:54:54.045 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2024-09-15T21:54:54.046 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQChV+dmjt6OLhAAHNy4JY7noAUWX46VVU4x+A== 2024-09-15T21:54:54.046 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2024-09-15T21:54:54.046 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCkV+dm8MyJLRAAD77YpqAIvPxkWuT2tI5GHw== 2024-09-15T21:54:54.046 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2024-09-15T21:54:54.046 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCnV+dmW/KTKxAA1OS3Aa+1bthnhiMXlb0rHA== 2024-09-15T21:54:54.046 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2024-09-15T21:54:54.046 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQCqV+dmMhzqKBAAJsxHgjP16Bi6sBoYRHLUnA== 2024-09-15T21:54:54.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:54:54.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:54:54.184 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:54:54.232 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:54:54.280 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:54:54.327 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:54:54.374 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T21:54:54.375 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T21:54:54.400 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:54:54.492 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T21:54:54.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T21:54:54.584 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:54:54.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T21:54:54.702 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T21:54:54.768 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:54:54.861 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T21:54:54.886 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:54:54.944 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T21:54:54.944 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:54:54.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:54:54.987 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:54:54.987 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:54:54.987 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:54:54.989 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:54:54.989 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:54:54.989 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:54:54.991 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:54:54.991 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T21:54:54.991 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T21:54:54.992 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T21:54:54.994 INFO:tasks.ceph.mgr.x:Started 2024-09-15T21:54:54.994 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-15T21:54:54.994 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-15T21:54:54.996 INFO:tasks.ceph.mgr.y:Started 2024-09-15T21:54:54.996 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T21:54:54.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T21:54:55.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.125+0000 7fc734e6f640 1 Processor -- start 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc734e6f640 1 -- start start 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc734e6f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc734e6f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc734e6f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc730090f70 con 0x7fc73012b120 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc734e6f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73008ef20 con 0x7fc7300959b0 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc734e6f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc730096280 con 0x7fc73008e610 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc72e59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc72ed9d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc73012b120 0x7fc73012d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47468/0 (socket says 172.21.15.47:47468) 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc72ed9d640 1 -- 172.21.15.47:0/1533381874 learned_addr learned my addr 172.21.15.47:0/1533381874 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:55.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 msgr2=0x7fc73008e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 msgr2=0x7fc73008e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.126+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.127+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.127+0000 7fc72dd9b640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 msgr2=0x7fc73008e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.127+0000 7fc72dd9b640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 msgr2=0x7fc73008e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.127+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:55.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.127+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:54:55.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.328+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:55.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.328+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:55.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.328+0000 7fc72dd9b640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 msgr2=0x7fc73008e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:54:55.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.328+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 msgr2=0x7fc73008e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:54:55.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.328+0000 7fc72dd9b640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 msgr2=0x7fc73008e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:55.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.328+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 msgr2=0x7fc73008e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:55.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.328+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:55.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.328+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:55.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.328+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:54:55.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.328+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:54:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.729+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.729+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.729+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 msgr2=0x7fc73008e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:54:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.729+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 msgr2=0x7fc73008e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.729+0000 7fc72dd9b640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 msgr2=0x7fc73008e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:54:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.729+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.729+0000 7fc72dd9b640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 msgr2=0x7fc73008e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.729+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.729+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:54:55.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:55.729+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:54:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:56.530+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:56.530+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:56.530+0000 7fc72dd9b640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 msgr2=0x7fc73008e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:54:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:56.530+0000 7fc72dd9b640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 msgr2=0x7fc73008e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:56.530+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:56.530+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 msgr2=0x7fc73008e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T21:54:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:56.530+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 msgr2=0x7fc73008e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:56.530+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:56.530+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:54:56.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:56.530+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:54:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc727fff640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 msgr2=0x7fc73008e9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:54:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc727fff640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc73008e9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:54:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc727fff640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 msgr2=0x7fc73008e0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:54:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc727fff640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc73008e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:54:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc727fff640 1 -- 172.21.15.47:0/1533381874 >> v1:172.21.15.47:6789/0 conn(0x7fc73012b120 legacy=0x7fc73012d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:54:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc727fff640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:54:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc727fff640 1 -- 172.21.15.47:0/1533381874 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73008ef20 con 0x7fc710003690 2024-09-15T21:54:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 msgr2=0x7fc710005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:54:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 msgr2=0x7fc710005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.127+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:54:58.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.328+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:58.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.328+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 msgr2=0x7fc710005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:54:58.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.328+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 msgr2=0x7fc710005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:58.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.328+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:58.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.328+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:54:58.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.729+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.729+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 msgr2=0x7fc710005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:54:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.729+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 msgr2=0x7fc710005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.729+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:58.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:58.729+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:54:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:59.530+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:54:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:59.530+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 msgr2=0x7fc710005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:54:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:59.530+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 msgr2=0x7fc710005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:54:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:59.530+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-15T21:54:59.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:54:59.530+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:55:00.418 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:00.416+0000 7efcff1bdec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T21:55:00.460 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:00.458+0000 7efcff1bdec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T21:55:00.511 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:00.510+0000 7efcff1bdec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T21:55:00.643 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:00.642+0000 7efcff1bdec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:55:00.753 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:00.752+0000 7efcff1bdec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T21:55:01.053 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:01.052+0000 7efcff1bdec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T21:55:01.112 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:01.110+0000 7efcff1bdec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T21:55:01.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc727fff640 1 -- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 msgr2=0x7fc710005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:55:01.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc727fff640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc727fff640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc7100083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc727fff640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc71000a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc727fff640 1 -- 172.21.15.47:0/1533381874 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73008fdc0 con 0x7fc73012b120 2024-09-15T21:55:01.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc727fff640 1 -- 172.21.15.47:0/1533381874 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73008ef20 con 0x7fc7300959b0 2024-09-15T21:55:01.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc727fff640 1 -- 172.21.15.47:0/1533381874 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc730096280 con 0x7fc73008e610 2024-09-15T21:55:01.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc7100083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc71000a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 msgr2=0x7fc71000a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:01.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc71000a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 >> v1:172.21.15.47:6789/0 conn(0x7fc73012b120 legacy=0x7fc71000cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc72e59c640 1 -- 172.21.15.47:0/1533381874 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71000f6d0 con 0x7fc7300959b0 2024-09-15T21:55:01.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.127+0000 7fc72d59a640 1 -- 172.21.15.47:0/1533381874 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843491211 0 0) 0x7fc73008fdc0 con 0x7fc73012b120 2024-09-15T21:55:01.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.128+0000 7fc72e59c640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc7100083d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc72000fe40 tx=0x7fc72000f7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cad8760567044ef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:01.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.128+0000 7fc72d59a640 1 -- 172.21.15.47:0/1533381874 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72002ed60 con 0x7fc7300959b0 2024-09-15T21:55:01.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.128+0000 7fc72d59a640 1 -- 172.21.15.47:0/1533381874 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc720017040 con 0x7fc7300959b0 2024-09-15T21:55:01.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.128+0000 7fc72d59a640 1 -- 172.21.15.47:0/1533381874 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720036640 con 0x7fc7300959b0 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.129+0000 7fc734e6f640 1 -- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 msgr2=0x7fc7100083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.129+0000 7fc734e6f640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc7100083d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc72000fe40 tx=0x7fc72000f7c0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.129+0000 7fc734e6f640 1 -- 172.21.15.47:0/1533381874 shutdown_connections 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.129+0000 7fc734e6f640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc73008e610 0x7fc71000a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.129+0000 7fc734e6f640 1 --2- 172.21.15.47:0/1533381874 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc7300959b0 0x7fc7100083d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.129+0000 7fc734e6f640 1 --2- 172.21.15.47:0/1533381874 >> v2:172.21.15.47:3300/0 conn(0x7fc710003690 0x7fc710005b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.129+0000 7fc734e6f640 1 -- 172.21.15.47:0/1533381874 >> 172.21.15.47:0/1533381874 conn(0x7fc730096aa0 msgr2=0x7fc730096ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.129+0000 7fc734e6f640 1 -- 172.21.15.47:0/1533381874 shutdown_connections 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.129+0000 7fc734e6f640 1 -- 172.21.15.47:0/1533381874 wait complete. 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.130+0000 7fc734e6f640 1 Processor -- start 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.130+0000 7fc734e6f640 1 -- start start 2024-09-15T21:55:01.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.130+0000 7fc734e6f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc710003690 0x7fc73008d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc734e6f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc73008e610 0x7fc73006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc734e6f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc7300959b0 0x7fc73006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc734e6f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73006f030 con 0x7fc710003690 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc734e6f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73006f400 con 0x7fc73008e610 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc734e6f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc730093770 con 0x7fc7300959b0 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72ed9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc7300959b0 0x7fc73006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72ed9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc7300959b0 0x7fc73006c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55662/0 (socket says 172.21.15.47:55662) 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72ed9d640 1 -- 172.21.15.47:0/897995878 learned_addr learned my addr 172.21.15.47:0/897995878 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72dd9b640 1 --2- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc73008e610 0x7fc73006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72e59c640 1 --2- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc710003690 0x7fc73008d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72ed9d640 1 -- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc73008e610 msgr2=0x7fc73006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72ed9d640 1 --2- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc73008e610 0x7fc73006bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72ed9d640 1 -- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc710003690 msgr2=0x7fc73008d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72ed9d640 1 --2- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc710003690 0x7fc73008d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72ed9d640 1 -- 172.21.15.47:0/897995878 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72000faa0 con 0x7fc7300959b0 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72e59c640 1 --2- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc710003690 0x7fc73008d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.131+0000 7fc72ed9d640 1 --2- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc7300959b0 0x7fc73006c1e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c820 tx=0x7fc7280015d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.132+0000 7fc7267fc640 1 -- 172.21.15.47:0/897995878 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72800ce10 con 0x7fc7300959b0 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.132+0000 7fc7267fc640 1 -- 172.21.15.47:0/897995878 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728004d30 con 0x7fc7300959b0 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.132+0000 7fc734e6f640 1 -- 172.21.15.47:0/897995878 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73006d700 con 0x7fc7300959b0 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.132+0000 7fc7267fc640 1 -- 172.21.15.47:0/897995878 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728004ea0 con 0x7fc7300959b0 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.132+0000 7fc734e6f640 1 -- 172.21.15.47:0/897995878 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73006c990 con 0x7fc7300959b0 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.132+0000 7fc7267fc640 1 -- 172.21.15.47:0/897995878 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc72800ab60 con 0x7fc7300959b0 2024-09-15T21:55:01.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.132+0000 7fc7267fc640 1 -- 172.21.15.47:0/897995878 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc728007990 con 0x7fc7300959b0 2024-09-15T21:55:01.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.133+0000 7fc734e6f640 1 -- 172.21.15.47:0/897995878 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73004bf60 con 0x7fc7300959b0 2024-09-15T21:55:01.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.135+0000 7fc7267fc640 1 -- 172.21.15.47:0/897995878 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fc728018070 con 0x7fc7300959b0 2024-09-15T21:55:01.172 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:01.171+0000 7f9ab8cddec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T21:55:01.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.197+0000 7fc734e6f640 1 -- 172.21.15.47:0/897995878 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fc730093550 con 0x7fc7300959b0 2024-09-15T21:55:01.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.201+0000 7fc7267fc640 1 -- 172.21.15.47:0/897995878 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc728012400 con 0x7fc7300959b0 2024-09-15T21:55:01.202 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-09-15T21:55:01.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.201+0000 7fc734e6f640 1 -- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc7300959b0 msgr2=0x7fc73006c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.201+0000 7fc734e6f640 1 --2- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc7300959b0 0x7fc73006c1e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c820 tx=0x7fc7280015d0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.202+0000 7fc734e6f640 1 -- 172.21.15.47:0/897995878 shutdown_connections 2024-09-15T21:55:01.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.202+0000 7fc734e6f640 1 --2- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc7300959b0 0x7fc73006c1e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.202+0000 7fc734e6f640 1 --2- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc73008e610 0x7fc73006bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.202+0000 7fc734e6f640 1 --2- 172.21.15.47:0/897995878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc710003690 0x7fc73008d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.202+0000 7fc734e6f640 1 -- 172.21.15.47:0/897995878 >> 172.21.15.47:0/897995878 conn(0x7fc730096aa0 msgr2=0x7fc730097db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:01.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.202+0000 7fc734e6f640 1 -- 172.21.15.47:0/897995878 shutdown_connections 2024-09-15T21:55:01.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.202+0000 7fc734e6f640 1 -- 172.21.15.47:0/897995878 wait complete. 2024-09-15T21:55:01.214 INFO:tasks.ceph:check_enable_crimson: False 2024-09-15T21:55:01.214 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T21:55:01.214 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:55:01.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T21:55:01.215 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:01.214+0000 7f9ab8cddec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T21:55:01.242 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:55:01.242 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T21:55:01.243 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:01.242+0000 7efcff1bdec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T21:55:01.268 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:01.266+0000 7f9ab8cddec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T21:55:01.269 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:55:01.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T21:55:01.337 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:55:01.337 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T21:55:01.350 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:01.348+0000 7f9ab8cddec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:55:01.375 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd new 3f514164-0bf2-4d96-99c2-dc948a0d3cb0 0 2024-09-15T21:55:01.446 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:01.445+0000 7f9ab8cddec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T21:55:01.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.511+0000 7f94df60f640 1 Processor -- start 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.512+0000 7f94df60f640 1 -- start start 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.512+0000 7f94df60f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d8095350 0x7f94d8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.512+0000 7f94df60f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a740 0x7f94d812e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.512+0000 7f94df60f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d8063d40 con 0x7f94d806a2f0 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.512+0000 7f94df60f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d806b2b0 con 0x7f94d8095350 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.512+0000 7f94df60f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8093cf0 con 0x7f94d806a740 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.512+0000 7f94dd3ab640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d8095350 0x7f94d8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.512+0000 7f94ddbac640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a740 0x7f94d812e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.512+0000 7f94dcbaa640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f94d806a2f0 0x7f94d812aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59988/0 (socket says 172.21.15.47:59988) 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.512+0000 7f94dcbaa640 1 -- 172.21.15.47:0/1999261482 learned_addr learned my addr 172.21.15.47:0/1999261482 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.513+0000 7f94cffff640 1 -- 172.21.15.47:0/1999261482 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056242522 0 0) 0x7f94d8063d40 con 0x7f94d806a2f0 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.513+0000 7f94dd3ab640 1 -- 172.21.15.47:0/1999261482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a740 msgr2=0x7f94d812e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.513+0000 7f94dd3ab640 1 --2- 172.21.15.47:0/1999261482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a740 0x7f94d812e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.513+0000 7f94dd3ab640 1 -- 172.21.15.47:0/1999261482 >> v1:172.21.15.47:6789/0 conn(0x7f94d806a2f0 legacy=0x7f94d812aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.513+0000 7f94dd3ab640 1 -- 172.21.15.47:0/1999261482 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8095d80 con 0x7f94d8095350 2024-09-15T21:55:01.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.513+0000 7f94ddbac640 1 --2- 172.21.15.47:0/1999261482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a740 0x7f94d812e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.513+0000 7f94dd3ab640 1 --2- 172.21.15.47:0/1999261482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d8095350 0x7f94d8069db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f94d400d900 tx=0x7f94d40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91cb82ae48b57309 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.514+0000 7f94cffff640 1 -- 172.21.15.47:0/1999261482 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4031510 con 0x7f94d8095350 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.514+0000 7f94cffff640 1 -- 172.21.15.47:0/1999261482 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d402f050 con 0x7f94d8095350 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.514+0000 7f94cffff640 1 -- 172.21.15.47:0/1999261482 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d40318a0 con 0x7f94d8095350 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.514+0000 7f94df60f640 1 -- 172.21.15.47:0/1999261482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d8095350 msgr2=0x7f94d8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.514+0000 7f94df60f640 1 --2- 172.21.15.47:0/1999261482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d8095350 0x7f94d8069db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f94d400d900 tx=0x7f94d40090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.514+0000 7f94df60f640 1 -- 172.21.15.47:0/1999261482 shutdown_connections 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.514+0000 7f94df60f640 1 --2- 172.21.15.47:0/1999261482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a740 0x7f94d812e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.514+0000 7f94df60f640 1 --2- 172.21.15.47:0/1999261482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d8095350 0x7f94d8069db0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.514+0000 7f94df60f640 1 -- 172.21.15.47:0/1999261482 >> 172.21.15.47:0/1999261482 conn(0x7f94d8096440 msgr2=0x7f94d8096840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.515+0000 7f94df60f640 1 -- 172.21.15.47:0/1999261482 shutdown_connections 2024-09-15T21:55:01.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.515+0000 7f94df60f640 1 -- 172.21.15.47:0/1999261482 wait complete. 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.515+0000 7f94df60f640 1 Processor -- start 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.515+0000 7f94df60f640 1 -- start start 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94df60f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a2f0 0x7f94d808f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94df60f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d806a740 0x7f94d808f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94df60f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d8095350 0x7f94d808fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94df60f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8092760 con 0x7f94d8095350 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94df60f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8092a70 con 0x7f94d806a740 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94df60f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8092d30 con 0x7f94d806a2f0 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94dd3ab640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a2f0 0x7f94d808f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94dd3ab640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a2f0 0x7f94d808f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55680/0 (socket says 172.21.15.47:55680) 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94dd3ab640 1 -- 172.21.15.47:0/1605284796 learned_addr learned my addr 172.21.15.47:0/1605284796 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94ddbac640 1 --2- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d8095350 0x7f94d808fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94dcbaa640 1 --2- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d806a740 0x7f94d808f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94dd3ab640 1 -- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d806a740 msgr2=0x7f94d808f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94dd3ab640 1 --2- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d806a740 0x7f94d808f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94dd3ab640 1 -- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d8095350 msgr2=0x7f94d808fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.516+0000 7f94dd3ab640 1 --2- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d8095350 0x7f94d808fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.517+0000 7f94dd3ab640 1 -- 172.21.15.47:0/1605284796 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d0009a20 con 0x7f94d806a2f0 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.517+0000 7f94dd3ab640 1 --2- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a2f0 0x7f94d808f2e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f94d400d3e0 tx=0x7f94d4037000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.517+0000 7f94cd7fa640 1 -- 172.21.15.47:0/1605284796 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4033890 con 0x7f94d806a2f0 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.517+0000 7f94cd7fa640 1 -- 172.21.15.47:0/1605284796 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d4042070 con 0x7f94d806a2f0 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.517+0000 7f94df60f640 1 -- 172.21.15.47:0/1605284796 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d400d410 con 0x7f94d806a2f0 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.517+0000 7f94cd7fa640 1 -- 172.21.15.47:0/1605284796 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4033c20 con 0x7f94d806a2f0 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.517+0000 7f94df60f640 1 -- 172.21.15.47:0/1605284796 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d8135110 con 0x7f94d806a2f0 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.517+0000 7f94cd7fa640 1 -- 172.21.15.47:0/1605284796 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f94d4044030 con 0x7f94d806a2f0 2024-09-15T21:55:01.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.517+0000 7f94cd7fa640 1 -- 172.21.15.47:0/1605284796 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f94d4047820 con 0x7f94d806a2f0 2024-09-15T21:55:01.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.518+0000 7f94df60f640 1 -- 172.21.15.47:0/1605284796 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d8091d30 con 0x7f94d806a2f0 2024-09-15T21:55:01.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.520+0000 7f94cd7fa640 1 -- 172.21.15.47:0/1605284796 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f94d4033d80 con 0x7f94d806a2f0 2024-09-15T21:55:01.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.589+0000 7f94df60f640 1 -- 172.21.15.47:0/1605284796 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd new", "uuid": "3f514164-0bf2-4d96-99c2-dc948a0d3cb0", "id": 0} v 0) v1 -- 0x7f94d804bf60 con 0x7f94d806a2f0 2024-09-15T21:55:01.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.593+0000 7f94cd7fa640 1 -- 172.21.15.47:0/1605284796 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3f514164-0bf2-4d96-99c2-dc948a0d3cb0", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f94d4047a30 con 0x7f94d806a2f0 2024-09-15T21:55:01.594 INFO:teuthology.orchestra.run.smithi047.stdout:0 2024-09-15T21:55:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.594+0000 7f94df60f640 1 -- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a2f0 msgr2=0x7f94d808f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.594+0000 7f94df60f640 1 --2- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a2f0 0x7f94d808f2e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f94d400d3e0 tx=0x7f94d4037000 comp rx=0 tx=0).stop 2024-09-15T21:55:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.594+0000 7f94df60f640 1 -- 172.21.15.47:0/1605284796 shutdown_connections 2024-09-15T21:55:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.594+0000 7f94df60f640 1 --2- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94d8095350 0x7f94d808fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.594+0000 7f94df60f640 1 --2- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f94d806a740 0x7f94d808f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.594+0000 7f94df60f640 1 --2- 172.21.15.47:0/1605284796 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f94d806a2f0 0x7f94d808f2e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.594+0000 7f94df60f640 1 -- 172.21.15.47:0/1605284796 >> 172.21.15.47:0/1605284796 conn(0x7f94d8096440 msgr2=0x7f94d806cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.594+0000 7f94df60f640 1 -- 172.21.15.47:0/1605284796 shutdown_connections 2024-09-15T21:55:01.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.594+0000 7f94df60f640 1 -- 172.21.15.47:0/1605284796 wait complete. 2024-09-15T21:55:01.597 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:01.596+0000 7efcff1bdec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T21:55:01.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd new 98cbb40c-7873-4bb4-8e27-42dd62549685 1 2024-09-15T21:55:01.653 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:01.652+0000 7efcff1bdec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T21:55:01.679 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:01.678+0000 7f9ab8cddec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T21:55:01.725 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:01.724+0000 7f9ab8cddec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T21:55:01.743 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:01.740+0000 7efcff1bdec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T21:55:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.744+0000 7fb077420640 1 Processor -- start 2024-09-15T21:55:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.744+0000 7fb077420640 1 -- start start 2024-09-15T21:55:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb077420640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070093670 0x7fb070093a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb077420640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb070095350 0x7fb070095720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb077420640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07006af80 con 0x7fb070095c60 2024-09-15T21:55:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb077420640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07006b1d0 con 0x7fb070095350 2024-09-15T21:55:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb077420640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb070063020 con 0x7fb070093670 2024-09-15T21:55:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb0751bc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070093670 0x7fb070093a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb0751bc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070093670 0x7fb070093a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44838/0 (socket says 172.21.15.47:44838) 2024-09-15T21:55:01.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb0749bb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb070095350 0x7fb070095720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb0751bc640 1 -- 172.21.15.47:0/53641925 learned_addr learned my addr 172.21.15.47:0/53641925 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb0751bc640 1 -- 172.21.15.47:0/53641925 >> v2:172.21.15.47:3300/0 conn(0x7fb070095350 msgr2=0x7fb070095720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb0751bc640 1 --2- 172.21.15.47:0/53641925 >> v2:172.21.15.47:3300/0 conn(0x7fb070095350 0x7fb070095720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb0751bc640 1 -- 172.21.15.47:0/53641925 >> v1:172.21.15.47:6789/0 conn(0x7fb070095c60 legacy=0x7fb070130b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb067fff640 1 -- 172.21.15.47:0/53641925 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532361307 0 0) 0x7fb07006af80 con 0x7fb070095c60 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb0751bc640 1 -- 172.21.15.47:0/53641925 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07006a140 con 0x7fb070093670 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.745+0000 7fb0749bb640 1 --2- 172.21.15.47:0/53641925 >> v2:172.21.15.47:3300/0 conn(0x7fb070095350 0x7fb070095720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.746+0000 7fb0751bc640 1 --2- 172.21.15.47:0/53641925 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070093670 0x7fb070093a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009910 tx=0x7fb06c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efb13ea57c4e9457 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.746+0000 7fb067fff640 1 -- 172.21.15.47:0/53641925 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c02ad20 con 0x7fb070093670 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.746+0000 7fb067fff640 1 -- 172.21.15.47:0/53641925 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c02ae80 con 0x7fb070093670 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.746+0000 7fb067fff640 1 -- 172.21.15.47:0/53641925 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c032670 con 0x7fb070093670 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.747+0000 7fb077420640 1 -- 172.21.15.47:0/53641925 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070093670 msgr2=0x7fb070093a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.747+0000 7fb077420640 1 --2- 172.21.15.47:0/53641925 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070093670 0x7fb070093a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009910 tx=0x7fb06c0091d0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.747+0000 7fb077420640 1 -- 172.21.15.47:0/53641925 shutdown_connections 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.747+0000 7fb077420640 1 --2- 172.21.15.47:0/53641925 >> v2:172.21.15.47:3300/0 conn(0x7fb070095350 0x7fb070095720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.747+0000 7fb077420640 1 --2- 172.21.15.47:0/53641925 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070093670 0x7fb070093a40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.747+0000 7fb077420640 1 -- 172.21.15.47:0/53641925 >> 172.21.15.47:0/53641925 conn(0x7fb070096440 msgr2=0x7fb070096840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.747+0000 7fb077420640 1 -- 172.21.15.47:0/53641925 shutdown_connections 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.747+0000 7fb077420640 1 -- 172.21.15.47:0/53641925 wait complete. 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.748+0000 7fb077420640 1 Processor -- start 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.748+0000 7fb077420640 1 -- start start 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb077420640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb070093670 0x7fb070131ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb077420640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070095350 0x7fb07011f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb077420640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070095c60 0x7fb07011fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb077420640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb070122210 con 0x7fb070095c60 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb077420640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0701229f0 con 0x7fb070095350 2024-09-15T21:55:01.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb077420640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb070122dc0 con 0x7fb070093670 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb0759bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070095c60 0x7fb07011fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb0751bc640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb070093670 0x7fb070131ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb0759bd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070095c60 0x7fb07011fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57032/0 (socket says 172.21.15.47:57032) 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb0759bd640 1 -- 172.21.15.47:0/3101637049 learned_addr learned my addr 172.21.15.47:0/3101637049 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.749+0000 7fb0749bb640 1 --2- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070095350 0x7fb07011f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0759bd640 1 -- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb070093670 msgr2=0x7fb070131ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0759bd640 1 --2- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb070093670 0x7fb070131ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0759bd640 1 -- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070095350 msgr2=0x7fb07011f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0759bd640 1 --2- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070095350 0x7fb07011f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0759bd640 1 -- 172.21.15.47:0/3101637049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb060009390 con 0x7fb070095c60 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0759bd640 1 --2- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070095c60 0x7fb07011fb50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb068001560 tx=0x7fb0680016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0657fa640 1 -- 172.21.15.47:0/3101637049 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06800a7f0 con 0x7fb070095c60 2024-09-15T21:55:01.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0657fa640 1 -- 172.21.15.47:0/3101637049 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06800a950 con 0x7fb070095c60 2024-09-15T21:55:01.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb077420640 1 -- 172.21.15.47:0/3101637049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06c009450 con 0x7fb070095c60 2024-09-15T21:55:01.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0657fa640 1 -- 172.21.15.47:0/3101637049 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06800ad50 con 0x7fb070095c60 2024-09-15T21:55:01.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb077420640 1 -- 172.21.15.47:0/3101637049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0701202b0 con 0x7fb070095c60 2024-09-15T21:55:01.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0657fa640 1 -- 172.21.15.47:0/3101637049 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fb068003030 con 0x7fb070095c60 2024-09-15T21:55:01.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.750+0000 7fb0657fa640 1 -- 172.21.15.47:0/3101637049 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fb06801b400 con 0x7fb070095c60 2024-09-15T21:55:01.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.751+0000 7fb077420640 1 -- 172.21.15.47:0/3101637049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07004bf60 con 0x7fb070095c60 2024-09-15T21:55:01.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.753+0000 7fb0657fa640 1 -- 172.21.15.47:0/3101637049 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fb068012630 con 0x7fb070095c60 2024-09-15T21:55:01.818 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:01.817+0000 7f9ab8cddec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T21:55:01.822 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:01.821+0000 7efcff1bdec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:55:01.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.822+0000 7fb077420640 1 -- 172.21.15.47:0/3101637049 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd new", "uuid": "98cbb40c-7873-4bb4-8e27-42dd62549685", "id": 1} v 0) v1 -- 0x7fb07008fa60 con 0x7fb070095c60 2024-09-15T21:55:01.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.825+0000 7fb0657fa640 1 -- 172.21.15.47:0/3101637049 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "98cbb40c-7873-4bb4-8e27-42dd62549685", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb06801b5c0 con 0x7fb070095c60 2024-09-15T21:55:01.827 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-09-15T21:55:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.826+0000 7fb077420640 1 -- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070095c60 msgr2=0x7fb07011fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.826+0000 7fb077420640 1 --2- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070095c60 0x7fb07011fb50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb068001560 tx=0x7fb0680016c0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.826+0000 7fb077420640 1 -- 172.21.15.47:0/3101637049 shutdown_connections 2024-09-15T21:55:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.826+0000 7fb077420640 1 --2- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb070095c60 0x7fb07011fb50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.826+0000 7fb077420640 1 --2- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb070095350 0x7fb07011f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.826+0000 7fb077420640 1 --2- 172.21.15.47:0/3101637049 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb070093670 0x7fb070131ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.826+0000 7fb077420640 1 -- 172.21.15.47:0/3101637049 >> 172.21.15.47:0/3101637049 conn(0x7fb070096440 msgr2=0x7fb07006dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.826+0000 7fb077420640 1 -- 172.21.15.47:0/3101637049 shutdown_connections 2024-09-15T21:55:01.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.826+0000 7fb077420640 1 -- 172.21.15.47:0/3101637049 wait complete. 2024-09-15T21:55:01.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd new cd82ae94-f43c-476f-bb97-a0699b859536 2 2024-09-15T21:55:01.871 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:01.870+0000 7efcff1bdec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T21:55:01.958 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:01.956+0000 7efcff1bdec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T21:55:01.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f86a4d9d640 1 Processor -- start 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f86a4d9d640 1 -- start start 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f86a4d9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86a0096cb0 0x7f86a008f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f86a4d9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a008f830 0x7f86a008fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f86a4d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86a012c420 0x7f86a012e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f86a4d9d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a00957c0 con 0x7f86a012c420 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f86a4d9d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0091090 con 0x7f86a0096cb0 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f86a4d9d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0090d80 con 0x7f86a008f830 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f8697fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a008f830 0x7f86a008fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f869ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86a012c420 0x7f86a012e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.967+0000 7f869e59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86a0096cb0 0x7f86a008f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.968+0000 7f869ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f86a012c420 0x7f86a012e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57038/0 (socket says 172.21.15.47:57038) 2024-09-15T21:55:01.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.968+0000 7f869ed9d640 1 -- 172.21.15.47:0/407527782 learned_addr learned my addr 172.21.15.47:0/407527782 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:01.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.968+0000 7f869ed9d640 1 -- 172.21.15.47:0/407527782 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a008f830 msgr2=0x7f86a008fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:01.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.968+0000 7f869ed9d640 1 --2- 172.21.15.47:0/407527782 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a008f830 0x7f86a008fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.968+0000 7f869ed9d640 1 -- 172.21.15.47:0/407527782 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86a0096cb0 msgr2=0x7f86a008f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:01.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.968+0000 7f869ed9d640 1 --2- 172.21.15.47:0/407527782 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86a0096cb0 0x7f86a008f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.968+0000 7f869ed9d640 1 -- 172.21.15.47:0/407527782 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a0091780 con 0x7f86a012c420 2024-09-15T21:55:01.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f869ed9d640 1 --2- 172.21.15.47:0/407527782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a012c420 0x7f86a012e800 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8698009f50 tx=0x7f8698009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2471b641b922dc1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f869dd9b640 1 -- 172.21.15.47:0/407527782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698031400 con 0x7f86a012c420 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f869dd9b640 1 -- 172.21.15.47:0/407527782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869802d050 con 0x7f86a012c420 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f869dd9b640 1 -- 172.21.15.47:0/407527782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86980317d0 con 0x7f86a012c420 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f86a4d9d640 1 -- 172.21.15.47:0/407527782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a012c420 msgr2=0x7f86a012e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f86a4d9d640 1 --2- 172.21.15.47:0/407527782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a012c420 0x7f86a012e800 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8698009f50 tx=0x7f8698009840 comp rx=0 tx=0).stop 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f86a4d9d640 1 -- 172.21.15.47:0/407527782 shutdown_connections 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f86a4d9d640 1 --2- 172.21.15.47:0/407527782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a012c420 0x7f86a012e800 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f86a4d9d640 1 --2- 172.21.15.47:0/407527782 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a008f830 0x7f86a008fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f86a4d9d640 1 --2- 172.21.15.47:0/407527782 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86a0096cb0 0x7f86a008f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.969+0000 7f86a4d9d640 1 -- 172.21.15.47:0/407527782 >> 172.21.15.47:0/407527782 conn(0x7f86a0097da0 msgr2=0x7f86a00981a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:01.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.970+0000 7f86a4d9d640 1 -- 172.21.15.47:0/407527782 shutdown_connections 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.970+0000 7f86a4d9d640 1 -- 172.21.15.47:0/407527782 wait complete. 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f86a4d9d640 1 Processor -- start 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f86a4d9d640 1 -- start start 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f86a4d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a008f830 0x7f86a0134bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f86a4d9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a0096cb0 0x7f86a0135130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f86a4d9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86a012c420 0x7f86a0135670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f86a4d9d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0088a90 con 0x7f86a008f830 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f86a4d9d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00887c0 con 0x7f86a012c420 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f86a4d9d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86a006dee0 con 0x7f86a0096cb0 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f8697fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a0096cb0 0x7f86a0135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f869e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a008f830 0x7f86a0134bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f869e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a008f830 0x7f86a0134bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57046/0 (socket says 172.21.15.47:57046) 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f8697fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a0096cb0 0x7f86a0135130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55720/0 (socket says 172.21.15.47:55720) 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.971+0000 7f869e59c640 1 -- 172.21.15.47:0/474785802 learned_addr learned my addr 172.21.15.47:0/474785802 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f869ed9d640 1 --2- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86a012c420 0x7f86a0135670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f869e59c640 1 -- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a0096cb0 msgr2=0x7f86a0135130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f869e59c640 1 --2- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a0096cb0 0x7f86a0135130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f869e59c640 1 -- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86a012c420 msgr2=0x7f86a0135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f869e59c640 1 --2- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86a012c420 0x7f86a0135670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f869e59c640 1 -- 172.21.15.47:0/474785802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8698009ae0 con 0x7f86a008f830 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f869e59c640 1 --2- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a008f830 0x7f86a0134bf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f869000b460 tx=0x7f8690008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f86967fc640 1 -- 172.21.15.47:0/474785802 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000d2a0 con 0x7f86a008f830 2024-09-15T21:55:01.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f86967fc640 1 -- 172.21.15.47:0/474785802 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869001b4a0 con 0x7f86a008f830 2024-09-15T21:55:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f86a4d9d640 1 -- 172.21.15.47:0/474785802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a00880e0 con 0x7f86a008f830 2024-09-15T21:55:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f86967fc640 1 -- 172.21.15.47:0/474785802 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869001b860 con 0x7f86a008f830 2024-09-15T21:55:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.972+0000 7f86a4d9d640 1 -- 172.21.15.47:0/474785802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a006cdc0 con 0x7f86a008f830 2024-09-15T21:55:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.973+0000 7f86967fc640 1 -- 172.21.15.47:0/474785802 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f8690005970 con 0x7f86a008f830 2024-09-15T21:55:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.973+0000 7f86967fc640 1 -- 172.21.15.47:0/474785802 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f8690006630 con 0x7f86a008f830 2024-09-15T21:55:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.974+0000 7f86a4d9d640 1 -- 172.21.15.47:0/474785802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a004bf60 con 0x7f86a008f830 2024-09-15T21:55:01.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:01.977+0000 7f86967fc640 1 -- 172.21.15.47:0/474785802 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f8690005c20 con 0x7f86a008f830 2024-09-15T21:55:02.016 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:02.014+0000 7efcff1bdec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T21:55:02.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.046+0000 7f86a4d9d640 1 -- 172.21.15.47:0/474785802 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cd82ae94-f43c-476f-bb97-a0699b859536", "id": 2} v 0) v1 -- 0x7f86a0064810 con 0x7f86a008f830 2024-09-15T21:55:02.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.049+0000 7f86967fc640 1 -- 172.21.15.47:0/474785802 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cd82ae94-f43c-476f-bb97-a0699b859536", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8690005e40 con 0x7f86a008f830 2024-09-15T21:55:02.051 INFO:teuthology.orchestra.run.smithi047.stdout:2 2024-09-15T21:55:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.050+0000 7f86a4d9d640 1 -- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a008f830 msgr2=0x7f86a0134bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.050+0000 7f86a4d9d640 1 --2- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a008f830 0x7f86a0134bf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f869000b460 tx=0x7f8690008040 comp rx=0 tx=0).stop 2024-09-15T21:55:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.050+0000 7f86a4d9d640 1 -- 172.21.15.47:0/474785802 shutdown_connections 2024-09-15T21:55:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.050+0000 7f86a4d9d640 1 --2- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f86a012c420 0x7f86a0135670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.050+0000 7f86a4d9d640 1 --2- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f86a0096cb0 0x7f86a0135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.050+0000 7f86a4d9d640 1 --2- 172.21.15.47:0/474785802 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f86a008f830 0x7f86a0134bf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.050+0000 7f86a4d9d640 1 -- 172.21.15.47:0/474785802 >> 172.21.15.47:0/474785802 conn(0x7f86a0097da0 msgr2=0x7f86a00947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.050+0000 7f86a4d9d640 1 -- 172.21.15.47:0/474785802 shutdown_connections 2024-09-15T21:55:02.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.050+0000 7f86a4d9d640 1 -- 172.21.15.47:0/474785802 wait complete. 2024-09-15T21:55:02.062 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd new 708e9506-4b05-4c61-95fa-a920cdc9d8fa 3 2024-09-15T21:55:02.064 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:02.062+0000 7efcff1bdec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T21:55:02.088 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.086+0000 7f9ab8cddec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T21:55:02.133 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.132+0000 7f9ab8cddec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T21:55:02.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.157+0000 7f7a9a900640 1 Processor -- start 2024-09-15T21:55:02.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.157+0000 7f7a9a900640 1 -- start start 2024-09-15T21:55:02.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a9a900640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408e1d0 0x7f7a9408e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:02.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a9a900640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a9408eae0 0x7f7a9412b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:02.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a9a900640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94097a20 con 0x7f7a9412b760 2024-09-15T21:55:02.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a9a900640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a94090520 con 0x7f7a9408eae0 2024-09-15T21:55:02.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a9a900640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94096a40 con 0x7f7a9408e1d0 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408e1d0 0x7f7a9408e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a937fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a9408eae0 0x7f7a9412b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408e1d0 0x7f7a9408e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44872/0 (socket says 172.21.15.47:44872) 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a937fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7a9408eae0 0x7f7a9412b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57050/0 (socket says 172.21.15.47:57050) 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a93fff640 1 -- 172.21.15.47:0/352254555 learned_addr learned my addr 172.21.15.47:0/352254555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a92ffd640 1 -- 172.21.15.47:0/352254555 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855775376 0 0) 0x7f7a94097a20 con 0x7f7a9412b760 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a937fe640 1 -- 172.21.15.47:0/352254555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408e1d0 msgr2=0x7f7a9408e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a937fe640 1 --2- 172.21.15.47:0/352254555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408e1d0 0x7f7a9408e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a937fe640 1 -- 172.21.15.47:0/352254555 >> v1:172.21.15.47:6789/0 conn(0x7f7a9412b760 legacy=0x7f7a9412db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a937fe640 1 -- 172.21.15.47:0/352254555 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a940964e0 con 0x7f7a9408eae0 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.158+0000 7f7a93fff640 1 --2- 172.21.15.47:0/352254555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408e1d0 0x7f7a9408e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:02.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.159+0000 7f7a937fe640 1 --2- 172.21.15.47:0/352254555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9408eae0 0x7f7a9412b220 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800bb20 tx=0x7f7a8800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6bc9e1697781fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:02.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.159+0000 7f7a92ffd640 1 -- 172.21.15.47:0/352254555 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88031510 con 0x7f7a9408eae0 2024-09-15T21:55:02.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.159+0000 7f7a92ffd640 1 -- 172.21.15.47:0/352254555 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8802f050 con 0x7f7a9408eae0 2024-09-15T21:55:02.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.159+0000 7f7a92ffd640 1 -- 172.21.15.47:0/352254555 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a880318c0 con 0x7f7a9408eae0 2024-09-15T21:55:02.162 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:02.160+0000 7efcff1bdec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T21:55:02.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.160+0000 7f7a9a900640 1 -- 172.21.15.47:0/352254555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9408eae0 msgr2=0x7f7a9412b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.160+0000 7f7a9a900640 1 --2- 172.21.15.47:0/352254555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9408eae0 0x7f7a9412b220 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800bb20 tx=0x7f7a8800b3b0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.160+0000 7f7a9a900640 1 -- 172.21.15.47:0/352254555 shutdown_connections 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.160+0000 7f7a9a900640 1 --2- 172.21.15.47:0/352254555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9408eae0 0x7f7a9412b220 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.160+0000 7f7a9a900640 1 --2- 172.21.15.47:0/352254555 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408e1d0 0x7f7a9408e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.160+0000 7f7a9a900640 1 -- 172.21.15.47:0/352254555 >> 172.21.15.47:0/352254555 conn(0x7f7a9409ab00 msgr2=0x7f7a94096ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.160+0000 7f7a9a900640 1 -- 172.21.15.47:0/352254555 shutdown_connections 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.160+0000 7f7a9a900640 1 -- 172.21.15.47:0/352254555 wait complete. 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.161+0000 7f7a9a900640 1 Processor -- start 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a9a900640 1 -- start start 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a9a900640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9408e1d0 0x7f7a9408d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a9a900640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408eae0 0x7f7a9408d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a9a900640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a9412b760 0x7f7a94089730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a9a900640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9408ded0 con 0x7f7a9408e1d0 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a9a900640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9408c490 con 0x7f7a9408eae0 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a9a900640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9408cd60 con 0x7f7a9412b760 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a937fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408eae0 0x7f7a9408d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9408e1d0 0x7f7a9408d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a937fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408eae0 0x7f7a9408d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44880/0 (socket says 172.21.15.47:44880) 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a937fe640 1 -- 172.21.15.47:0/145317109 learned_addr learned my addr 172.21.15.47:0/145317109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.162+0000 7f7a98e9d640 1 --2- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a9412b760 0x7f7a94089730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a93fff640 1 -- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a9412b760 msgr2=0x7f7a94089730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a93fff640 1 --2- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a9412b760 0x7f7a94089730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a93fff640 1 -- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408eae0 msgr2=0x7f7a9408d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a93fff640 1 --2- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408eae0 0x7f7a9408d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a93fff640 1 -- 172.21.15.47:0/145317109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84009450 con 0x7f7a9408e1d0 2024-09-15T21:55:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a937fe640 1 --2- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408eae0 0x7f7a9408d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a93fff640 1 --2- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9408e1d0 0x7f7a9408d450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400ea20 tx=0x7f7a840091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a7bfff640 1 -- 172.21.15.47:0/145317109 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84009680 con 0x7f7a9408e1d0 2024-09-15T21:55:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a7bfff640 1 -- 172.21.15.47:0/145317109 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8400ca10 con 0x7f7a9408e1d0 2024-09-15T21:55:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a7bfff640 1 -- 172.21.15.47:0/145317109 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8400cdd0 con 0x7f7a9408e1d0 2024-09-15T21:55:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a9a900640 1 -- 172.21.15.47:0/145317109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8800b630 con 0x7f7a9408e1d0 2024-09-15T21:55:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a9a900640 1 -- 172.21.15.47:0/145317109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9408c990 con 0x7f7a9408e1d0 2024-09-15T21:55:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.163+0000 7f7a7bfff640 1 -- 172.21.15.47:0/145317109 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f7a84005970 con 0x7f7a9408e1d0 2024-09-15T21:55:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.164+0000 7f7a7bfff640 1 -- 172.21.15.47:0/145317109 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f7a84013e40 con 0x7f7a9408e1d0 2024-09-15T21:55:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.164+0000 7f7a9a900640 1 -- 172.21.15.47:0/145317109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9404bf60 con 0x7f7a9408e1d0 2024-09-15T21:55:02.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.168+0000 7f7a7bfff640 1 -- 172.21.15.47:0/145317109 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f7a84020060 con 0x7f7a9408e1d0 2024-09-15T21:55:02.214 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:02.212+0000 7efcff1bdec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T21:55:02.221 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.220+0000 7f9ab8cddec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T21:55:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.237+0000 7f7a9a900640 1 -- 172.21.15.47:0/145317109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd new", "uuid": "708e9506-4b05-4c61-95fa-a920cdc9d8fa", "id": 3} v 0) v1 -- 0x7f7a94064760 con 0x7f7a9408e1d0 2024-09-15T21:55:02.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.239+0000 7f7a7bfff640 1 -- 172.21.15.47:0/145317109 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "708e9506-4b05-4c61-95fa-a920cdc9d8fa", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7a84005cc0 con 0x7f7a9408e1d0 2024-09-15T21:55:02.241 INFO:teuthology.orchestra.run.smithi047.stdout:3 2024-09-15T21:55:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.240+0000 7f7a9a900640 1 -- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9408e1d0 msgr2=0x7f7a9408d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.240+0000 7f7a9a900640 1 --2- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9408e1d0 0x7f7a9408d450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400ea20 tx=0x7f7a840091d0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.240+0000 7f7a9a900640 1 -- 172.21.15.47:0/145317109 shutdown_connections 2024-09-15T21:55:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.240+0000 7f7a9a900640 1 --2- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7a9412b760 0x7f7a94089730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.240+0000 7f7a9a900640 1 --2- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7a9408eae0 0x7f7a9408d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.240+0000 7f7a9a900640 1 --2- 172.21.15.47:0/145317109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a9408e1d0 0x7f7a9408d450 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.240+0000 7f7a9a900640 1 -- 172.21.15.47:0/145317109 >> 172.21.15.47:0/145317109 conn(0x7f7a9409ab00 msgr2=0x7f7a94092db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.240+0000 7f7a9a900640 1 -- 172.21.15.47:0/145317109 shutdown_connections 2024-09-15T21:55:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.241+0000 7f7a9a900640 1 -- 172.21.15.47:0/145317109 wait complete. 2024-09-15T21:55:02.252 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T21:55:02.252 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T21:55:02.270 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:02.267+0000 7efcff1bdec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T21:55:02.271 INFO:tasks.ceph.osd.0:Started 2024-09-15T21:55:02.272 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T21:55:02.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T21:55:02.274 INFO:tasks.ceph.osd.1:Started 2024-09-15T21:55:02.274 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T21:55:02.274 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T21:55:02.277 INFO:tasks.ceph.osd.2:Started 2024-09-15T21:55:02.277 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T21:55:02.277 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T21:55:02.281 INFO:tasks.ceph.osd.3:Started 2024-09-15T21:55:02.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:55:02.320 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.318+0000 7f9ab8cddec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:55:02.380 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:02.378+0000 7efcff1bdec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T21:55:02.393 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.392+0000 7f9ab8cddec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T21:55:02.427 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:02.426+0000 7efcff1bdec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T21:55:02.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.430+0000 7f4fd0234640 1 Processor -- start 2024-09-15T21:55:02.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fd0234640 1 -- start start 2024-09-15T21:55:02.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fd0234640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4fc812a340 0x7f4fc812a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fd0234640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc8096900 0x7f4fc8096d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fd0234640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc806af80 con 0x7f4fc812ac50 2024-09-15T21:55:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fd0234640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc8069a30 con 0x7f4fc812a340 2024-09-15T21:55:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fd0234640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc806a720 con 0x7f4fc8096900 2024-09-15T21:55:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fce7d1640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc8096900 0x7f4fc8096d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fcdfd0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4fc812a340 0x7f4fc812a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fce7d1640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc8096900 0x7f4fc8096d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55798/0 (socket says 172.21.15.47:55798) 2024-09-15T21:55:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fce7d1640 1 -- 172.21.15.47:0/2632890849 learned_addr learned my addr 172.21.15.47:0/2632890849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.431+0000 7f4fcd7cf640 1 --1- 172.21.15.47:0/2632890849 >> v1:172.21.15.47:6789/0 conn(0x7f4fc812ac50 0x7f4fc80961f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60032/0 (socket says 172.21.15.47:60032) 2024-09-15T21:55:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.432+0000 7f4fcdfd0640 1 -- 172.21.15.47:0/2632890849 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc8096900 msgr2=0x7f4fc8096d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.432+0000 7f4fcdfd0640 1 --2- 172.21.15.47:0/2632890849 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc8096900 0x7f4fc8096d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.432+0000 7f4fcdfd0640 1 -- 172.21.15.47:0/2632890849 >> v1:172.21.15.47:6789/0 conn(0x7f4fc812ac50 legacy=0x7f4fc80961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.432+0000 7f4fccfce640 1 -- 172.21.15.47:0/2632890849 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675393185 0 0) 0x7f4fc806af80 con 0x7f4fc812ac50 2024-09-15T21:55:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.432+0000 7f4fcdfd0640 1 -- 172.21.15.47:0/2632890849 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc806abb0 con 0x7f4fc812a340 2024-09-15T21:55:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.432+0000 7f4fcdfd0640 1 --2- 172.21.15.47:0/2632890849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc812a340 0x7f4fc812a710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009e60 tx=0x7f4fc4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129ba93588959eb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.433+0000 7f4fccfce640 1 -- 172.21.15.47:0/2632890849 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4033980 con 0x7f4fc812a340 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.433+0000 7f4fccfce640 1 -- 172.21.15.47:0/2632890849 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc4033ae0 con 0x7f4fc812a340 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.433+0000 7f4fccfce640 1 -- 172.21.15.47:0/2632890849 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4033e80 con 0x7f4fc812a340 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.433+0000 7f4fd0234640 1 -- 172.21.15.47:0/2632890849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc812a340 msgr2=0x7f4fc812a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.433+0000 7f4fd0234640 1 --2- 172.21.15.47:0/2632890849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc812a340 0x7f4fc812a710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009e60 tx=0x7f4fc4009780 comp rx=0 tx=0).stop 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.433+0000 7f4fd0234640 1 -- 172.21.15.47:0/2632890849 shutdown_connections 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.433+0000 7f4fd0234640 1 --2- 172.21.15.47:0/2632890849 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc8096900 0x7f4fc8096d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.433+0000 7f4fd0234640 1 --2- 172.21.15.47:0/2632890849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc812a340 0x7f4fc812a710 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.433+0000 7f4fd0234640 1 -- 172.21.15.47:0/2632890849 >> 172.21.15.47:0/2632890849 conn(0x7f4fc809b740 msgr2=0x7f4fc809bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.434+0000 7f4fd0234640 1 -- 172.21.15.47:0/2632890849 shutdown_connections 2024-09-15T21:55:02.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.434+0000 7f4fd0234640 1 -- 172.21.15.47:0/2632890849 wait complete. 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.434+0000 7f4fd0234640 1 Processor -- start 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.434+0000 7f4fd0234640 1 -- start start 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fd0234640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc8096900 0x7f4fc8087f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fd0234640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fc812a340 0x7f4fc80884b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fd0234640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc812ac50 0x7f4fc80889f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fd0234640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc808dec0 con 0x7f4fc8096900 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fd0234640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc808ad50 con 0x7f4fc812a340 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fd0234640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc808b160 con 0x7f4fc812ac50 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fcdfd0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc8096900 0x7f4fc8087f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fcdfd0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc8096900 0x7f4fc8087f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57132/0 (socket says 172.21.15.47:57132) 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fcdfd0640 1 -- 172.21.15.47:0/2139288015 learned_addr learned my addr 172.21.15.47:0/2139288015 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fcd7cf640 1 --2- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fc812a340 0x7f4fc80884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fce7d1640 1 --2- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc812ac50 0x7f4fc80889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fcdfd0640 1 -- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc812ac50 msgr2=0x7f4fc80889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fcdfd0640 1 --2- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc812ac50 0x7f4fc80889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fcdfd0640 1 -- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fc812a340 msgr2=0x7f4fc80884b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fcdfd0640 1 --2- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fc812a340 0x7f4fc80884b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.435+0000 7f4fcdfd0640 1 -- 172.21.15.47:0/2139288015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc4009a20 con 0x7f4fc8096900 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.436+0000 7f4fcdfd0640 1 --2- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc8096900 0x7f4fc8087f70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40332b0 tx=0x7f4fc403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.436+0000 7f4fbdffb640 1 -- 172.21.15.47:0/2139288015 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc403a6b0 con 0x7f4fc8096900 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.436+0000 7f4fbdffb640 1 -- 172.21.15.47:0/2139288015 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc403a830 con 0x7f4fc8096900 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.436+0000 7f4fbdffb640 1 -- 172.21.15.47:0/2139288015 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc403abf0 con 0x7f4fc8096900 2024-09-15T21:55:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.436+0000 7f4fd0234640 1 -- 172.21.15.47:0/2139288015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc8089d60 con 0x7f4fc8096900 2024-09-15T21:55:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.437+0000 7f4fd0234640 1 -- 172.21.15.47:0/2139288015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc80890d0 con 0x7f4fc8096900 2024-09-15T21:55:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.437+0000 7f4fbdffb640 1 -- 172.21.15.47:0/2139288015 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f4fc4042050 con 0x7f4fc8096900 2024-09-15T21:55:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.437+0000 7f4fbdffb640 1 -- 172.21.15.47:0/2139288015 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f4fc404ee20 con 0x7f4fc8096900 2024-09-15T21:55:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.438+0000 7f4fd0234640 1 -- 172.21.15.47:0/2139288015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc804bf60 con 0x7f4fc8096900 2024-09-15T21:55:02.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.440+0000 7f4fbdffb640 1 -- 172.21.15.47:0/2139288015 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f4fc4053060 con 0x7f4fc8096900 2024-09-15T21:55:02.473 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:02.472+0000 7efcff1bdec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T21:55:02.486 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.485+0000 7f9ab8cddec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T21:55:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.498+0000 7f4fd0234640 1 -- 172.21.15.47:0/2139288015 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fc80646b0 con 0x7f4fc8096900 2024-09-15T21:55:02.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.498+0000 7f4fbdffb640 1 -- 172.21.15.47:0/2139288015 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7f4fc4049030 con 0x7f4fc8096900 2024-09-15T21:55:02.500 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:55:02.500 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:55:02.238284+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:55:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.499+0000 7f4fd0234640 1 -- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc8096900 msgr2=0x7f4fc8087f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.499+0000 7f4fd0234640 1 --2- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc8096900 0x7f4fc8087f70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40332b0 tx=0x7f4fc403e000 comp rx=0 tx=0).stop 2024-09-15T21:55:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.499+0000 7f4fd0234640 1 -- 172.21.15.47:0/2139288015 shutdown_connections 2024-09-15T21:55:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.499+0000 7f4fd0234640 1 --2- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4fc812ac50 0x7f4fc80889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.499+0000 7f4fd0234640 1 --2- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4fc812a340 0x7f4fc80884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.499+0000 7f4fd0234640 1 --2- 172.21.15.47:0/2139288015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fc8096900 0x7f4fc8087f70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.499+0000 7f4fd0234640 1 -- 172.21.15.47:0/2139288015 >> 172.21.15.47:0/2139288015 conn(0x7f4fc809b740 msgr2=0x7f4fc8094b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.499+0000 7f4fd0234640 1 -- 172.21.15.47:0/2139288015 shutdown_connections 2024-09-15T21:55:02.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:02.499+0000 7f4fd0234640 1 -- 172.21.15.47:0/2139288015 wait complete. 2024-09-15T21:55:02.511 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T21:55:02.511 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T21:55:02.540 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.539+0000 7f9ab8cddec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T21:55:02.582 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.580+0000 7f9ab8cddec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T21:55:02.665 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.664+0000 7f9ab8cddec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T21:55:02.689 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:02.687+0000 7efcff1bdec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T21:55:02.719 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.718+0000 7f9ab8cddec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T21:55:02.770 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.768+0000 7f9ab8cddec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T21:55:02.822 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.821+0000 7f9ab8cddec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T21:55:02.864 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.862+0000 7f9ab8cddec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T21:55:02.883 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-09-15T21:55:02.881+0000 7f625d723440 -1 Falling back to public interface 2024-09-15T21:55:02.894 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-09-15T21:55:02.893+0000 7f755d323440 -1 Falling back to public interface 2024-09-15T21:55:02.905 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:02.904+0000 7f9ab8cddec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T21:55:02.907 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-09-15T21:55:02.906+0000 7f25e8123440 -1 Falling back to public interface 2024-09-15T21:55:02.915 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:55:02.917 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-09-15T21:55:02.915+0000 7f5be5211440 -1 Falling back to public interface 2024-09-15T21:55:03.002 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.000+0000 7eff60a4f640 1 Processor -- start 2024-09-15T21:55:03.002 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.001+0000 7eff60a4f640 1 -- start start 2024-09-15T21:55:03.003 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.001+0000 7eff60a4f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c0941e0 0x7eff5c095a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:03.003 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.001+0000 7eff60a4f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c096360 0x7eff5c12f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:03.003 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.001+0000 7eff60a4f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5c097700 con 0x7eff5c095f50 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.001+0000 7eff60a4f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c091590 con 0x7eff5c0941e0 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.001+0000 7eff60a4f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c0967a0 con 0x7eff5c096360 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.001+0000 7eff5a59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c0941e0 0x7eff5c095a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.001+0000 7eff5ad9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c096360 0x7eff5c12f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.001+0000 7eff59d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7eff5c095f50 0x7eff5c12c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60038/0 (socket says 172.21.15.47:60038) 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.001+0000 7eff59d9b640 1 -- 172.21.15.47:0/3148727446 learned_addr learned my addr 172.21.15.47:0/3148727446 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.002+0000 7eff5a59c640 1 -- 172.21.15.47:0/3148727446 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c096360 msgr2=0x7eff5c12f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.002+0000 7eff5a59c640 1 --2- 172.21.15.47:0/3148727446 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c096360 0x7eff5c12f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.002+0000 7eff5a59c640 1 -- 172.21.15.47:0/3148727446 >> v1:172.21.15.47:6789/0 conn(0x7eff5c095f50 legacy=0x7eff5c12c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.002+0000 7eff5959a640 1 -- 172.21.15.47:0/3148727446 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991737637 0 0) 0x7eff5c097700 con 0x7eff5c095f50 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.002+0000 7eff5a59c640 1 -- 172.21.15.47:0/3148727446 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c068ee0 con 0x7eff5c0941e0 2024-09-15T21:55:03.004 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.002+0000 7eff5ad9d640 1 --2- 172.21.15.47:0/3148727446 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c096360 0x7eff5c12f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.002+0000 7eff5a59c640 1 --2- 172.21.15.47:0/3148727446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c0941e0 0x7eff5c095a10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009880 tx=0x7eff4c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1b920c65b4ec36 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff5959a640 1 -- 172.21.15.47:0/3148727446 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c02ac50 con 0x7eff5c0941e0 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff5959a640 1 -- 172.21.15.47:0/3148727446 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4c02adb0 con 0x7eff5c0941e0 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff5959a640 1 -- 172.21.15.47:0/3148727446 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c032670 con 0x7eff5c0941e0 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff60a4f640 1 -- 172.21.15.47:0/3148727446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c0941e0 msgr2=0x7eff5c095a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff60a4f640 1 --2- 172.21.15.47:0/3148727446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c0941e0 0x7eff5c095a10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009880 tx=0x7eff4c0090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff60a4f640 1 -- 172.21.15.47:0/3148727446 shutdown_connections 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff60a4f640 1 --2- 172.21.15.47:0/3148727446 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c096360 0x7eff5c12f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff60a4f640 1 --2- 172.21.15.47:0/3148727446 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c0941e0 0x7eff5c095a10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff60a4f640 1 -- 172.21.15.47:0/3148727446 >> 172.21.15.47:0/3148727446 conn(0x7eff5c09aa20 msgr2=0x7eff5c096b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff60a4f640 1 -- 172.21.15.47:0/3148727446 shutdown_connections 2024-09-15T21:55:03.008 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.003+0000 7eff60a4f640 1 -- 172.21.15.47:0/3148727446 wait complete. 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.004+0000 7eff60a4f640 1 Processor -- start 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.004+0000 7eff60a4f640 1 -- start start 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff60a4f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c0941e0 0x7eff5c08d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff60a4f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c095f50 0x7eff5c08d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff60a4f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff5c096360 0x7eff5c08e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff60a4f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c08c830 con 0x7eff5c096360 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff60a4f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c08cdd0 con 0x7eff5c095f50 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff60a4f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c08d090 con 0x7eff5c0941e0 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff5a59c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c0941e0 0x7eff5c08d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff59d9b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c095f50 0x7eff5c08d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff5ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff5c096360 0x7eff5c08e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff5a59c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c0941e0 0x7eff5c08d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55882/0 (socket says 172.21.15.47:55882) 2024-09-15T21:55:03.009 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff5a59c640 1 -- 172.21.15.47:0/155387886 learned_addr learned my addr 172.21.15.47:0/155387886 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff59d9b640 1 -- 172.21.15.47:0/155387886 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c0941e0 msgr2=0x7eff5c08d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff59d9b640 1 --2- 172.21.15.47:0/155387886 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c0941e0 0x7eff5c08d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff59d9b640 1 -- 172.21.15.47:0/155387886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff5c096360 msgr2=0x7eff5c08e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff59d9b640 1 --2- 172.21.15.47:0/155387886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff5c096360 0x7eff5c08e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff59d9b640 1 -- 172.21.15.47:0/155387886 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff54009a20 con 0x7eff5c095f50 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff59d9b640 1 --2- 172.21.15.47:0/155387886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c095f50 0x7eff5c08d890 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7eff50000c00 tx=0x7eff50007c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff4a7fc640 1 -- 172.21.15.47:0/155387886 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50005aa0 con 0x7eff5c095f50 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff4a7fc640 1 -- 172.21.15.47:0/155387886 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff50005c60 con 0x7eff5c095f50 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff60a4f640 1 -- 172.21.15.47:0/155387886 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4c009390 con 0x7eff5c095f50 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff4a7fc640 1 -- 172.21.15.47:0/155387886 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50007320 con 0x7eff5c095f50 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.005+0000 7eff60a4f640 1 -- 172.21.15.47:0/155387886 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c08fac0 con 0x7eff5c095f50 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.006+0000 7eff4a7fc640 1 -- 172.21.15.47:0/155387886 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 2) v1 ==== 29207+0+0 (secure 0 0 0) 0x7eff5001b3b0 con 0x7eff5c095f50 2024-09-15T21:55:03.010 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.006+0000 7eff4a7fc640 1 -- 172.21.15.47:0/155387886 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7eff50035dc0 con 0x7eff5c095f50 2024-09-15T21:55:03.011 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.007+0000 7eff60a4f640 1 -- 172.21.15.47:0/155387886 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5c04bf60 con 0x7eff5c095f50 2024-09-15T21:55:03.011 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.009+0000 7eff4a7fc640 1 -- 172.21.15.47:0/155387886 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7eff50017030 con 0x7eff5c095f50 2024-09-15T21:55:03.068 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.066+0000 7eff60a4f640 1 -- 172.21.15.47:0/155387886 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff5c08c440 con 0x7eff5c095f50 2024-09-15T21:55:03.068 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.067+0000 7eff4a7fc640 1 -- 172.21.15.47:0/155387886 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7eff5001a360 con 0x7eff5c095f50 2024-09-15T21:55:03.068 INFO:teuthology.misc.health.smithi047.stdout: 2024-09-15T21:55:03.069 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":6,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:55:02.238284+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:55:03.069 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.068+0000 7eff60a4f640 1 -- 172.21.15.47:0/155387886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c095f50 msgr2=0x7eff5c08d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:03.070 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.068+0000 7eff60a4f640 1 --2- 172.21.15.47:0/155387886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c095f50 0x7eff5c08d890 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7eff50000c00 tx=0x7eff50007c80 comp rx=0 tx=0).stop 2024-09-15T21:55:03.070 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.068+0000 7eff60a4f640 1 -- 172.21.15.47:0/155387886 shutdown_connections 2024-09-15T21:55:03.070 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.068+0000 7eff60a4f640 1 --2- 172.21.15.47:0/155387886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7eff5c096360 0x7eff5c08e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:03.070 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.068+0000 7eff60a4f640 1 --2- 172.21.15.47:0/155387886 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7eff5c095f50 0x7eff5c08d890 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:03.070 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.068+0000 7eff60a4f640 1 --2- 172.21.15.47:0/155387886 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7eff5c0941e0 0x7eff5c08d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:03.070 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.068+0000 7eff60a4f640 1 -- 172.21.15.47:0/155387886 >> 172.21.15.47:0/155387886 conn(0x7eff5c09aa20 msgr2=0x7eff5c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:03.070 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.068+0000 7eff60a4f640 1 -- 172.21.15.47:0/155387886 shutdown_connections 2024-09-15T21:55:03.070 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:03.068+0000 7eff60a4f640 1 -- 172.21.15.47:0/155387886 wait complete. 2024-09-15T21:55:03.078 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-15T21:55:03.112 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:55:03.111+0000 7f9ab8cddec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T21:55:03.701 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:03.700+0000 7efce80ec640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T21:55:03.701 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:55:03.700+0000 7efce80ec640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T21:55:06.305 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-09-15T21:55:06.303+0000 7f625d723440 -1 osd.0 0 log_to_monitors true 2024-09-15T21:55:06.348 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-09-15T21:55:06.346+0000 7f755d323440 -1 osd.1 0 log_to_monitors true 2024-09-15T21:55:06.379 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-09-15T21:55:06.378+0000 7f5be5211440 -1 osd.2 0 log_to_monitors true 2024-09-15T21:55:06.395 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-09-15T21:55:06.394+0000 7f25e8123440 -1 osd.3 0 log_to_monitors true 2024-09-15T21:55:09.248 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-09-15T21:55:09.246+0000 7f25e2bf6640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T21:55:09.256 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-09-15T21:55:09.255+0000 7f25dbbe8640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:55:09.319 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-09-15T21:55:09.318+0000 7f7557df6640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T21:55:09.324 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-09-15T21:55:09.322+0000 7f7550de8640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:55:09.324 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-09-15T21:55:09.323+0000 7f5bdf9f6640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T21:55:09.329 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-09-15T21:55:09.328+0000 7f5bd89e8640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:55:09.334 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-09-15T21:55:09.332+0000 7f62581f6640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T21:55:09.338 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-09-15T21:55:09.336+0000 7f62511e8640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:55:09.481 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:55:09.569 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.567+0000 7fded0b9d640 1 Processor -- start 2024-09-15T21:55:09.569 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.567+0000 7fded0b9d640 1 -- start start 2024-09-15T21:55:09.576 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fded0b9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 0x7fdecc126960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:09.576 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fded0b9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 0x7fdecc12e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:09.576 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fded0b9d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdecc092b00 con 0x7fdecc126ea0 2024-09-15T21:55:09.576 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fded0b9d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc091750 con 0x7fdecc126590 2024-09-15T21:55:09.576 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fded0b9d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc091990 con 0x7fdecc090bd0 2024-09-15T21:55:09.576 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fdeca59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 0x7fdecc126960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fdec9d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdecc126ea0 0x7fdecc0904c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:54986/0 (socket says 172.21.15.47:54986) 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fdecad9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 0x7fdecc12e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fdec9d9b640 1 -- 172.21.15.47:0/1335041980 learned_addr learned my addr 172.21.15.47:0/1335041980 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fdec959a640 1 -- 172.21.15.47:0/1335041980 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044501300 0 0) 0x7fdecc092b00 con 0x7fdecc126ea0 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fdeca59c640 1 -- 172.21.15.47:0/1335041980 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 msgr2=0x7fdecc12e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fdeca59c640 1 --2- 172.21.15.47:0/1335041980 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 0x7fdecc12e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fdeca59c640 1 -- 172.21.15.47:0/1335041980 >> v1:172.21.15.47:6789/0 conn(0x7fdecc126ea0 legacy=0x7fdecc0904c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fdeca59c640 1 -- 172.21.15.47:0/1335041980 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc0954a0 con 0x7fdecc126590 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.568+0000 7fdecad9d640 1 --2- 172.21.15.47:0/1335041980 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 0x7fdecc12e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.569+0000 7fdeca59c640 1 --2- 172.21.15.47:0/1335041980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 0x7fdecc126960 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00b930 tx=0x7fdebc0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30978f081a012574 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.569+0000 7fdec959a640 1 -- 172.21.15.47:0/1335041980 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc02f510 con 0x7fdecc126590 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.569+0000 7fdec959a640 1 -- 172.21.15.47:0/1335041980 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc02d050 con 0x7fdecc126590 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.569+0000 7fdec959a640 1 -- 172.21.15.47:0/1335041980 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc02f8a0 con 0x7fdecc126590 2024-09-15T21:55:09.577 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.570+0000 7fded0b9d640 1 -- 172.21.15.47:0/1335041980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 msgr2=0x7fdecc126960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.570+0000 7fded0b9d640 1 --2- 172.21.15.47:0/1335041980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 0x7fdecc126960 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00b930 tx=0x7fdebc0090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.570+0000 7fded0b9d640 1 -- 172.21.15.47:0/1335041980 shutdown_connections 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.570+0000 7fded0b9d640 1 --2- 172.21.15.47:0/1335041980 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 0x7fdecc12e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.570+0000 7fded0b9d640 1 --2- 172.21.15.47:0/1335041980 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 0x7fdecc126960 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.570+0000 7fded0b9d640 1 -- 172.21.15.47:0/1335041980 >> 172.21.15.47:0/1335041980 conn(0x7fdecc099c20 msgr2=0x7fdecc095ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.570+0000 7fded0b9d640 1 -- 172.21.15.47:0/1335041980 shutdown_connections 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.570+0000 7fded0b9d640 1 -- 172.21.15.47:0/1335041980 wait complete. 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fded0b9d640 1 Processor -- start 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fded0b9d640 1 -- start start 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fded0b9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 0x7fdecc135400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fded0b9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 0x7fdecc135940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fded0b9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc126ea0 0x7fdecc135e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fded0b9d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc124740 con 0x7fdecc126ea0 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fded0b9d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc1254e0 con 0x7fdecc126590 2024-09-15T21:55:09.578 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fded0b9d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc098410 con 0x7fdecc090bd0 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fdeca59c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 0x7fdecc135400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fdec9d9b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 0x7fdecc135940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fdecad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc126ea0 0x7fdecc135e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fdeca59c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 0x7fdecc135400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:34088/0 (socket says 172.21.15.47:34088) 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fdeca59c640 1 -- 172.21.15.47:0/1473581345 learned_addr learned my addr 172.21.15.47:0/1473581345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.571+0000 7fdecad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc126ea0 0x7fdecc135e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45330/0 (socket says 172.21.15.47:45330) 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fdec9d9b640 1 -- 172.21.15.47:0/1473581345 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 msgr2=0x7fdecc135400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fdec9d9b640 1 --2- 172.21.15.47:0/1473581345 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 0x7fdecc135400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fdec9d9b640 1 -- 172.21.15.47:0/1473581345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc126ea0 msgr2=0x7fdecc135e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fdec9d9b640 1 --2- 172.21.15.47:0/1473581345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc126ea0 0x7fdecc135e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fdec9d9b640 1 -- 172.21.15.47:0/1473581345 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec400de30 con 0x7fdecc126590 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fdec9d9b640 1 --2- 172.21.15.47:0/1473581345 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 0x7fdecc135940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000c00 tx=0x7fdec0007d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fdeba7fc640 1 -- 172.21.15.47:0/1473581345 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00062e0 con 0x7fdecc126590 2024-09-15T21:55:09.579 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fdeba7fc640 1 -- 172.21.15.47:0/1473581345 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec0006460 con 0x7fdecc126590 2024-09-15T21:55:09.580 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fded0b9d640 1 -- 172.21.15.47:0/1473581345 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc00b440 con 0x7fdecc126590 2024-09-15T21:55:09.580 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fdeba7fc640 1 -- 172.21.15.47:0/1473581345 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00049f0 con 0x7fdecc126590 2024-09-15T21:55:09.580 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.572+0000 7fded0b9d640 1 -- 172.21.15.47:0/1473581345 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc121fe0 con 0x7fdecc126590 2024-09-15T21:55:09.580 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.573+0000 7fded0b9d640 1 -- 172.21.15.47:0/1473581345 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc04bf60 con 0x7fdecc126590 2024-09-15T21:55:09.580 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.573+0000 7fdeba7fc640 1 -- 172.21.15.47:0/1473581345 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 4) v1 ==== 58495+0+0 (secure 0 0 0) 0x7fdec001b3b0 con 0x7fdecc126590 2024-09-15T21:55:09.580 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.573+0000 7fdeba7fc640 1 -- 172.21.15.47:0/1473581345 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1509+0+0 (secure 0 0 0) 0x7fdec0062080 con 0x7fdecc126590 2024-09-15T21:55:09.580 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.577+0000 7fdeba7fc640 1 -- 172.21.15.47:0/1473581345 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fdec003d710 con 0x7fdecc126590 2024-09-15T21:55:09.718 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.717+0000 7fded0b9d640 1 -- 172.21.15.47:0/1473581345 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdecc093420 con 0x7fdecc126590 2024-09-15T21:55:09.719 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.717+0000 7fdeba7fc640 1 -- 172.21.15.47:0/1473581345 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+4108 (secure 0 0 0) 0x7fdec0017070 con 0x7fdecc126590 2024-09-15T21:55:09.719 INFO:teuthology.misc.health.smithi047.stdout: 2024-09-15T21:55:09.719 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":8,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:55:07.704765+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:55:09.721 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.719+0000 7fded0b9d640 1 -- 172.21.15.47:0/1473581345 >> v1:172.21.15.47:6800/32267 conn(0x7fdeac04cae0 legacy=0x7fdeac04ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:09.721 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.719+0000 7fded0b9d640 1 -- 172.21.15.47:0/1473581345 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 msgr2=0x7fdecc135940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:09.721 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.719+0000 7fded0b9d640 1 --2- 172.21.15.47:0/1473581345 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 0x7fdecc135940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdec0000c00 tx=0x7fdec0007d20 comp rx=0 tx=0).stop 2024-09-15T21:55:09.721 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.719+0000 7fded0b9d640 1 -- 172.21.15.47:0/1473581345 shutdown_connections 2024-09-15T21:55:09.722 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.719+0000 7fded0b9d640 1 --2- 172.21.15.47:0/1473581345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdecc126ea0 0x7fdecc135e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:09.722 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.719+0000 7fded0b9d640 1 --2- 172.21.15.47:0/1473581345 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdecc126590 0x7fdecc135940 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:09.722 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.719+0000 7fded0b9d640 1 --2- 172.21.15.47:0/1473581345 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdecc090bd0 0x7fdecc135400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:09.722 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.719+0000 7fded0b9d640 1 -- 172.21.15.47:0/1473581345 >> 172.21.15.47:0/1473581345 conn(0x7fdecc099c20 msgr2=0x7fdecc096de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:09.722 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.719+0000 7fded0b9d640 1 -- 172.21.15.47:0/1473581345 shutdown_connections 2024-09-15T21:55:09.722 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:09.719+0000 7fded0b9d640 1 -- 172.21.15.47:0/1473581345 wait complete. 2024-09-15T21:55:09.730 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-15T21:55:16.133 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:55:16.234 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9ecef640 1 Processor -- start 2024-09-15T21:55:16.234 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9ecef640 1 -- start start 2024-09-15T21:55:16.234 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9ecef640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 0x7fab980692a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9ecef640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fab980697e0 0x7fab98069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9ecef640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab9812a190 0x7fab9812a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9ecef640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9806aa00 con 0x7fab980697e0 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9ecef640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9806a130 con 0x7fab98096a20 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9ecef640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab980972f0 con 0x7fab9812a190 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab97fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fab980697e0 0x7fab98069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9d28c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab9812a190 0x7fab9812a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9ca8b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 0x7fab980692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab97fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fab980697e0 0x7fab98069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45352/0 (socket says 172.21.15.47:45352) 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab97fff640 1 -- 172.21.15.47:0/3748676363 learned_addr learned my addr 172.21.15.47:0/3748676363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.231+0000 7fab9d28c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab9812a190 0x7fab9812a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:34118/0 (socket says 172.21.15.47:34118) 2024-09-15T21:55:16.235 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.232+0000 7fab97fff640 1 -- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab9812a190 msgr2=0x7fab9812a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:16.236 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.232+0000 7fab97fff640 1 --2- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab9812a190 0x7fab9812a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.236 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.232+0000 7fab97fff640 1 -- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 msgr2=0x7fab980692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:16.236 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.232+0000 7fab97fff640 1 --2- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 0x7fab980692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.236 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.232+0000 7fab97fff640 1 -- 172.21.15.47:0/3748676363 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9808f280 con 0x7fab980697e0 2024-09-15T21:55:16.236 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.232+0000 7fab9d28c640 1 --2- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab9812a190 0x7fab9812a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:16.236 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.232+0000 7fab9ca8b640 1 --2- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 0x7fab980692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:16.239 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.232+0000 7fab97fff640 1 --2- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab980697e0 0x7fab98069bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fab84009970 tx=0x7fab840091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856d4201fa5e6efc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:16.239 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab977fe640 1 -- 172.21.15.47:0/3748676363 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8402aca0 con 0x7fab980697e0 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab977fe640 1 -- 172.21.15.47:0/3748676363 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8402ae00 con 0x7fab980697e0 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab977fe640 1 -- 172.21.15.47:0/3748676363 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84032670 con 0x7fab980697e0 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab9ecef640 1 -- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab980697e0 msgr2=0x7fab98069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab9ecef640 1 --2- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab980697e0 0x7fab98069bf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fab84009970 tx=0x7fab840091d0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab9ecef640 1 -- 172.21.15.47:0/3748676363 shutdown_connections 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab9ecef640 1 --2- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab9812a190 0x7fab9812a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab9ecef640 1 --2- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab980697e0 0x7fab98069bf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab9ecef640 1 --2- 172.21.15.47:0/3748676363 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 0x7fab980692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab9ecef640 1 -- 172.21.15.47:0/3748676363 >> 172.21.15.47:0/3748676363 conn(0x7fab9809bc10 msgr2=0x7fab98097b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab9ecef640 1 -- 172.21.15.47:0/3748676363 shutdown_connections 2024-09-15T21:55:16.240 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.233+0000 7fab9ecef640 1 -- 172.21.15.47:0/3748676363 wait complete. 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.234+0000 7fab9ecef640 1 Processor -- start 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.234+0000 7fab9ecef640 1 -- start start 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9ecef640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab980697e0 0x7fab981392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9ecef640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 0x7fab98139810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9ecef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab9812a190 0x7fab98139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9ecef640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab981390f0 con 0x7fab9812a190 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9ecef640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9808e3a0 con 0x7fab98096a20 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9ecef640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fab9808ea30 con 0x7fab980697e0 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9ca8b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab980697e0 0x7fab981392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab97fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 0x7fab98139810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9d28c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab9812a190 0x7fab98139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9ca8b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab980697e0 0x7fab981392d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:34124/0 (socket says 172.21.15.47:34124) 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9ca8b640 1 -- 172.21.15.47:0/3871278165 learned_addr learned my addr 172.21.15.47:0/3871278165 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:16.241 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9d28c640 1 -- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab980697e0 msgr2=0x7fab981392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9d28c640 1 --2- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab980697e0 0x7fab981392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9d28c640 1 -- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 msgr2=0x7fab98139810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9d28c640 1 --2- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 0x7fab98139810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9d28c640 1 -- 172.21.15.47:0/3871278165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c009390 con 0x7fab9812a190 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.235+0000 7fab9ca8b640 1 --2- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab980697e0 0x7fab981392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.236+0000 7fab9d28c640 1 --2- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab9812a190 0x7fab98139d50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fab90009f10 tx=0x7fab9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.236+0000 7fab94ff9640 1 -- 172.21.15.47:0/3871278165 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9000cdb0 con 0x7fab9812a190 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.236+0000 7fab94ff9640 1 -- 172.21.15.47:0/3871278165 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9000f040 con 0x7fab9812a190 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.236+0000 7fab9ecef640 1 -- 172.21.15.47:0/3871278165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab90009a20 con 0x7fab9812a190 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.236+0000 7fab94ff9640 1 -- 172.21.15.47:0/3871278165 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab900146d0 con 0x7fab9812a190 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.236+0000 7fab9ecef640 1 -- 172.21.15.47:0/3871278165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab84009450 con 0x7fab9812a190 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.237+0000 7fab9ecef640 1 -- 172.21.15.47:0/3871278165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9804bf60 con 0x7fab9812a190 2024-09-15T21:55:16.242 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.237+0000 7fab94ff9640 1 -- 172.21.15.47:0/3871278165 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fab90003020 con 0x7fab9812a190 2024-09-15T21:55:16.243 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.237+0000 7fab94ff9640 1 -- 172.21.15.47:0/3871278165 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2455+0+0 (secure 0 0 0) 0x7fab90063a30 con 0x7fab9812a190 2024-09-15T21:55:16.243 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.241+0000 7fab94ff9640 1 -- 172.21.15.47:0/3871278165 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fab9003ed00 con 0x7fab9812a190 2024-09-15T21:55:16.383 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.380+0000 7fab9ecef640 1 -- 172.21.15.47:0/3871278165 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab980646a0 con 0x7fab9812a190 2024-09-15T21:55:16.383 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.381+0000 7fab94ff9640 1 -- 172.21.15.47:0/3871278165 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+6543 (secure 0 0 0) 0x7fab90018070 con 0x7fab9812a190 2024-09-15T21:55:16.383 INFO:teuthology.misc.health.smithi047.stdout: 2024-09-15T21:55:16.383 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":12,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:55:13.256240+0000","last_up_change":"2024-09-15T21:55:10.248544+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":32760}]},"public_addr":"172.21.15.47:6801/32760","cluster_addr":"172.21.15.47:6802/32760","heartbeat_back_addr":"172.21.15.47:6804/32760","heartbeat_front_addr":"172.21.15.47:6803/32760","state":["exists","up"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":32761}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":32761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":32761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":32761}]},"public_addr":"172.21.15.47:6805/32761","cluster_addr":"172.21.15.47:6806/32761","heartbeat_back_addr":"172.21.15.47:6808/32761","heartbeat_front_addr":"172.21.15.47:6807/32761","state":["exists","up"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":32772}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":32772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":32772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":32772}]},"public_addr":"172.21.15.47:6813/32772","cluster_addr":"172.21.15.47:6814/32772","heartbeat_back_addr":"172.21.15.47:6816/32772","heartbeat_front_addr":"172.21.15.47:6815/32772","state":["exists","up"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":32770}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":32770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":32770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":32770}]},"public_addr":"172.21.15.47:6809/32770","cluster_addr":"172.21.15.47:6810/32770","heartbeat_back_addr":"172.21.15.47:6812/32770","heartbeat_front_addr":"172.21.15.47:6811/32770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:55:16.385 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.382+0000 7fab9ecef640 1 -- 172.21.15.47:0/3871278165 >> v1:172.21.15.47:6800/32267 conn(0x7fab7c04cb70 legacy=0x7fab7c04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:16.385 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.382+0000 7fab9ecef640 1 -- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab9812a190 msgr2=0x7fab98139d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:16.385 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.382+0000 7fab9ecef640 1 --2- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab9812a190 0x7fab98139d50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fab90009f10 tx=0x7fab9000b040 comp rx=0 tx=0).stop 2024-09-15T21:55:16.385 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.383+0000 7fab9ecef640 1 -- 172.21.15.47:0/3871278165 shutdown_connections 2024-09-15T21:55:16.386 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.383+0000 7fab9ecef640 1 --2- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab9812a190 0x7fab98139d50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.386 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.383+0000 7fab9ecef640 1 --2- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fab98096a20 0x7fab98139810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.386 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.383+0000 7fab9ecef640 1 --2- 172.21.15.47:0/3871278165 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fab980697e0 0x7fab981392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.386 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.383+0000 7fab9ecef640 1 -- 172.21.15.47:0/3871278165 >> 172.21.15.47:0/3871278165 conn(0x7fab9809bc10 msgr2=0x7fab98099cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:16.386 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.383+0000 7fab9ecef640 1 -- 172.21.15.47:0/3871278165 shutdown_connections 2024-09-15T21:55:16.386 INFO:teuthology.misc.health.smithi047.stderr:2024-09-15T21:55:16.383+0000 7fab9ecef640 1 -- 172.21.15.47:0/3871278165 wait complete. 2024-09-15T21:55:16.394 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-15T21:55:16.394 INFO:tasks.ceph:Creating RBD pool 2024-09-15T21:55:16.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T21:55:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.488+0000 7fc8f240f640 1 Processor -- start 2024-09-15T21:55:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.488+0000 7fc8f240f640 1 -- start start 2024-09-15T21:55:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.488+0000 7fc8f240f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8ec096130 0x7fc8ec12f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.488+0000 7fc8f240f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.488+0000 7fc8f240f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ec0969d0 con 0x7fc8ec0695b0 2024-09-15T21:55:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.488+0000 7fc8f240f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ec096c70 con 0x7fc8ec096130 2024-09-15T21:55:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.488+0000 7fc8f240f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec096f30 con 0x7fc8ec08d390 2024-09-15T21:55:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.488+0000 7fc8eb7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8ec096130 0x7fc8ec12f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.488+0000 7fc8f09ac640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.489+0000 7fc8eb7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc8ec096130 0x7fc8ec12f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45370/0 (socket says 172.21.15.47:45370) 2024-09-15T21:55:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.489+0000 7fc8ebfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc8ec0695b0 0x7fc8ec069980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55002/0 (socket says 172.21.15.47:55002) 2024-09-15T21:55:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.489+0000 7fc8eb7fe640 1 -- 172.21.15.47:0/793632520 learned_addr learned my addr 172.21.15.47:0/793632520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.489+0000 7fc8f09ac640 1 -- 172.21.15.47:0/793632520 >> v2:172.21.15.47:3300/0 conn(0x7fc8ec096130 msgr2=0x7fc8ec12f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.489+0000 7fc8f09ac640 1 --2- 172.21.15.47:0/793632520 >> v2:172.21.15.47:3300/0 conn(0x7fc8ec096130 0x7fc8ec12f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.489+0000 7fc8f09ac640 1 -- 172.21.15.47:0/793632520 >> v1:172.21.15.47:6789/0 conn(0x7fc8ec0695b0 legacy=0x7fc8ec069980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.489+0000 7fc8f09ac640 1 -- 172.21.15.47:0/793632520 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec132d00 con 0x7fc8ec08d390 2024-09-15T21:55:16.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.489+0000 7fc8eaffd640 1 -- 172.21.15.47:0/793632520 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132521371 0 0) 0x7fc8ec0969d0 con 0x7fc8ec0695b0 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.489+0000 7fc8eb7fe640 1 --2- 172.21.15.47:0/793632520 >> v2:172.21.15.47:3300/0 conn(0x7fc8ec096130 0x7fc8ec12f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8f09ac640 1 --2- 172.21.15.47:0/793632520 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1317b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009e60 tx=0x7fc8e4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89bc479ba9139e46 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8eaffd640 1 -- 172.21.15.47:0/793632520 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007550 con 0x7fc8ec08d390 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8eaffd640 1 -- 172.21.15.47:0/793632520 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e402d050 con 0x7fc8ec08d390 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8eaffd640 1 -- 172.21.15.47:0/793632520 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40078e0 con 0x7fc8ec08d390 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8f240f640 1 -- 172.21.15.47:0/793632520 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 msgr2=0x7fc8ec1317b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8f240f640 1 --2- 172.21.15.47:0/793632520 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1317b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4009e60 tx=0x7fc8e4009780 comp rx=0 tx=0).stop 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8f240f640 1 -- 172.21.15.47:0/793632520 shutdown_connections 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8f240f640 1 --2- 172.21.15.47:0/793632520 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1317b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8f240f640 1 --2- 172.21.15.47:0/793632520 >> v2:172.21.15.47:3300/0 conn(0x7fc8ec096130 0x7fc8ec12f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8f240f640 1 -- 172.21.15.47:0/793632520 >> 172.21.15.47:0/793632520 conn(0x7fc8ec08ed10 msgr2=0x7fc8ec09b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:16.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8f240f640 1 -- 172.21.15.47:0/793632520 shutdown_connections 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.490+0000 7fc8f240f640 1 -- 172.21.15.47:0/793632520 wait complete. 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.491+0000 7fc8f240f640 1 Processor -- start 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.491+0000 7fc8f240f640 1 -- start start 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8f240f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8ec0695b0 0x7fc8ec1392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8f240f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1397f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8f240f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8ec096130 0x7fc8ec06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8f240f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec132e90 con 0x7fc8ec096130 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8f240f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec09a7c0 con 0x7fc8ec0695b0 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8f240f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec120f00 con 0x7fc8ec08d390 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8eb7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8ebfff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8ec0695b0 0x7fc8ec1392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8f09ac640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8ec096130 0x7fc8ec06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:16.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8eb7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1397f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:34136/0 (socket says 172.21.15.47:34136) 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8eb7fe640 1 -- 172.21.15.47:0/956541235 learned_addr learned my addr 172.21.15.47:0/956541235 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8ebfff640 1 -- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 msgr2=0x7fc8ec1397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8ebfff640 1 --2- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8ebfff640 1 -- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8ec096130 msgr2=0x7fc8ec06d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8ebfff640 1 --2- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8ec096130 0x7fc8ec06d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8ebfff640 1 -- 172.21.15.47:0/956541235 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e0009450 con 0x7fc8ec0695b0 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8eb7fe640 1 --2- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8f09ac640 1 --2- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8ec096130 0x7fc8ec06d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8ebfff640 1 --2- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8ec0695b0 0x7fc8ec1392b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00c4e0 tx=0x7fc8dc007ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8d3fff640 1 -- 172.21.15.47:0/956541235 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0072e0 con 0x7fc8ec0695b0 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8d3fff640 1 -- 172.21.15.47:0/956541235 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc007440 con 0x7fc8ec0695b0 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8d3fff640 1 -- 172.21.15.47:0/956541235 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0047a0 con 0x7fc8ec0695b0 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.492+0000 7fc8f240f640 1 -- 172.21.15.47:0/956541235 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e4009a20 con 0x7fc8ec0695b0 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.493+0000 7fc8f240f640 1 -- 172.21.15.47:0/956541235 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec139d30 con 0x7fc8ec0695b0 2024-09-15T21:55:16.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.494+0000 7fc8d3fff640 1 -- 172.21.15.47:0/956541235 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fc8dc004bd0 con 0x7fc8ec0695b0 2024-09-15T21:55:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.494+0000 7fc8f240f640 1 -- 172.21.15.47:0/956541235 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec04bf60 con 0x7fc8ec0695b0 2024-09-15T21:55:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.494+0000 7fc8d3fff640 1 -- 172.21.15.47:0/956541235 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2455+0+0 (secure 0 0 0) 0x7fc8dc0624b0 con 0x7fc8ec0695b0 2024-09-15T21:55:16.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.498+0000 7fc8d3fff640 1 -- 172.21.15.47:0/956541235 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fc8dc03d680 con 0x7fc8ec0695b0 2024-09-15T21:55:16.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:16.634+0000 7fc8f240f640 1 -- 172.21.15.47:0/956541235 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fc8ec120660 con 0x7fc8ec0695b0 2024-09-15T21:55:17.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.269+0000 7fc8d3fff640 1 -- 172.21.15.47:0/956541235 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (secure 0 0 0) 0x7fc8dc040e90 con 0x7fc8ec0695b0 2024-09-15T21:55:17.272 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'rbd' created 2024-09-15T21:55:17.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.272+0000 7fc8f240f640 1 -- 172.21.15.47:0/956541235 >> v1:172.21.15.47:6800/32267 conn(0x7fc8d804cc20 legacy=0x7fc8d804f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:17.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.272+0000 7fc8f240f640 1 -- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8ec0695b0 msgr2=0x7fc8ec1392b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:17.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.272+0000 7fc8f240f640 1 --2- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8ec0695b0 0x7fc8ec1392b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00c4e0 tx=0x7fc8dc007ca0 comp rx=0 tx=0).stop 2024-09-15T21:55:17.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.272+0000 7fc8f240f640 1 -- 172.21.15.47:0/956541235 shutdown_connections 2024-09-15T21:55:17.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.272+0000 7fc8f240f640 1 --2- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8ec096130 0x7fc8ec06d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:17.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.272+0000 7fc8f240f640 1 --2- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc8ec08d390 0x7fc8ec1397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:17.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.272+0000 7fc8f240f640 1 --2- 172.21.15.47:0/956541235 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc8ec0695b0 0x7fc8ec1392b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:17.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.272+0000 7fc8f240f640 1 -- 172.21.15.47:0/956541235 >> 172.21.15.47:0/956541235 conn(0x7fc8ec08ed10 msgr2=0x7fc8ec0945b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:17.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.273+0000 7fc8f240f640 1 -- 172.21.15.47:0/956541235 shutdown_connections 2024-09-15T21:55:17.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.273+0000 7fc8f240f640 1 -- 172.21.15.47:0/956541235 wait complete. 2024-09-15T21:55:17.287 DEBUG:teuthology.orchestra.run.smithi047:> rbd --cluster ceph pool init rbd 2024-09-15T21:55:17.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.328+0000 7faaa802fb80 1 Processor -- start 2024-09-15T21:55:17.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.328+0000 7faaa802fb80 1 -- start start 2024-09-15T21:55:17.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa802fb80 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c467483d80 0x55c46748c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:17.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa802fb80 1 --2- >> v2:172.21.15.47:3300/0 conn(0x55c46748cb60 0x55c467535280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:17.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa802fb80 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x55c4673c40e0 con 0x55c46748c790 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa802fb80 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c4673fd260 con 0x55c46748cb60 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa802fb80 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c4673f8f40 con 0x55c467483d80 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa5fa3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x55c46748cb60 0x55c467535280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa57a2640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c467483d80 0x55c46748c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa57a2640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c467483d80 0x55c46748c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:34146/0 (socket says 172.21.15.47:34146) 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa5fa3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x55c46748cb60 0x55c467535280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45388/0 (socket says 172.21.15.47:45388) 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa57a2640 1 -- 172.21.15.47:0/2618389813 learned_addr learned my addr 172.21.15.47:0/2618389813 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa4fa1640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x55c46748c790 0x55c467531b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:55004/0 (socket says 172.21.15.47:55004) 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa5fa3640 1 -- 172.21.15.47:0/2618389813 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c467483d80 msgr2=0x55c46748c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa5fa3640 1 --2- 172.21.15.47:0/2618389813 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c467483d80 0x55c46748c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa5fa3640 1 -- 172.21.15.47:0/2618389813 >> v1:172.21.15.47:6789/0 conn(0x55c46748c790 legacy=0x55c467531b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:17.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faa97fff640 1 -- 172.21.15.47:0/2618389813 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982797639 0 0) 0x55c4673c40e0 con 0x55c46748c790 2024-09-15T21:55:17.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa5fa3640 1 -- 172.21.15.47:0/2618389813 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c46739d270 con 0x55c46748cb60 2024-09-15T21:55:17.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.329+0000 7faaa57a2640 1 --2- 172.21.15.47:0/2618389813 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c467483d80 0x55c46748c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.331+0000 7faaa5fa3640 1 --2- 172.21.15.47:0/2618389813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c46748cb60 0x55c467535280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faaa000a5a0 tx=0x7faaa0045a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af75f0261201b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.331+0000 7faa97fff640 1 -- 172.21.15.47:0/2618389813 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0052420 con 0x55c46748cb60 2024-09-15T21:55:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.331+0000 7faa97fff640 1 -- 172.21.15.47:0/2618389813 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa0052580 con 0x55c46748cb60 2024-09-15T21:55:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.331+0000 7faa97fff640 1 -- 172.21.15.47:0/2618389813 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0052730 con 0x55c46748cb60 2024-09-15T21:55:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.332+0000 7faaa802fb80 1 -- 172.21.15.47:0/2618389813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c46748cb60 msgr2=0x55c467535280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.332+0000 7faaa802fb80 1 --2- 172.21.15.47:0/2618389813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c46748cb60 0x55c467535280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faaa000a5a0 tx=0x7faaa0045a10 comp rx=0 tx=0).stop 2024-09-15T21:55:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.332+0000 7faaa802fb80 1 -- 172.21.15.47:0/2618389813 shutdown_connections 2024-09-15T21:55:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.332+0000 7faaa802fb80 1 --2- 172.21.15.47:0/2618389813 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c46748cb60 0x55c467535280 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.332+0000 7faaa802fb80 1 --2- 172.21.15.47:0/2618389813 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c467483d80 0x55c46748c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.332+0000 7faaa802fb80 1 -- 172.21.15.47:0/2618389813 >> 172.21.15.47:0/2618389813 conn(0x55c467485e00 msgr2=0x55c4674882f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:17.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.332+0000 7faaa802fb80 1 -- 172.21.15.47:0/2618389813 shutdown_connections 2024-09-15T21:55:17.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.332+0000 7faaa802fb80 1 -- 172.21.15.47:0/2618389813 wait complete. 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa802fb80 1 Processor -- start 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa802fb80 1 -- start start 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa802fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c467483d80 0x55c46752e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa802fb80 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55c46748c790 0x55c46752e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa802fb80 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c46748cb60 0x55c467499a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa802fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4673bbba0 con 0x55c467483d80 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa802fb80 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4673bb840 con 0x55c46748c790 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa802fb80 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x55c46737e7a0 con 0x55c46748cb60 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa57a2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c467483d80 0x55c46752e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa4fa1640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55c46748c790 0x55c46752e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa5fa3640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c46748cb60 0x55c467499a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:17.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.333+0000 7faaa57a2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c467483d80 0x55c46752e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45394/0 (socket says 172.21.15.47:45394) 2024-09-15T21:55:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faaa57a2640 1 -- 172.21.15.47:0/1515228584 learned_addr learned my addr 172.21.15.47:0/1515228584 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faaa57a2640 1 -- 172.21.15.47:0/1515228584 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c46748cb60 msgr2=0x55c467499a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faaa57a2640 1 --2- 172.21.15.47:0/1515228584 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x55c46748cb60 0x55c467499a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faaa57a2640 1 -- 172.21.15.47:0/1515228584 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55c46748c790 msgr2=0x55c46752e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faaa57a2640 1 --2- 172.21.15.47:0/1515228584 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x55c46748c790 0x55c46752e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faaa57a2640 1 -- 172.21.15.47:0/1515228584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa98005db0 con 0x55c467483d80 2024-09-15T21:55:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faaa57a2640 1 --2- 172.21.15.47:0/1515228584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c467483d80 0x55c46752e3a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa98008af0 tx=0x7faa9800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9800f040 con 0x55c467483d80 2024-09-15T21:55:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa000a0e0 con 0x55c467483d80 2024-09-15T21:55:17.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9800d980 con 0x55c467483d80 2024-09-15T21:55:17.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9800dd20 con 0x55c467483d80 2024-09-15T21:55:17.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.334+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4673aa0e0 con 0x55c467483d80 2024-09-15T21:55:17.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.335+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7faa98013610 con 0x55c467483d80 2024-09-15T21:55:17.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.336+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2797+0+0 (secure 0 0 0) 0x7faa980627d0 con 0x55c467483d80 2024-09-15T21:55:17.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:17.336+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x55c46739d600 con 0x55c467483d80 2024-09-15T21:55:18.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.269+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa9803d950 con 0x55c467483d80 2024-09-15T21:55:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.270+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x55c4673bb630 con 0x55c467483d80 2024-09-15T21:55:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.270+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa9803d950 con 0x55c467483d80 2024-09-15T21:55:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.270+0000 7faa977fe640 1 -- 172.21.15.47:0/1515228584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=14}) v3 -- 0x7faa80002990 con 0x55c467483d80 2024-09-15T21:55:18.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.271+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 642+0+0 (secure 0 0 0) 0x7faa98041500 con 0x55c467483d80 2024-09-15T21:55:18.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.272+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 --> v1:172.21.15.47:6809/32770 -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14) v8 -- 0x55c4675462c0 con 0x55c46749ab00 2024-09-15T21:55:18.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.279+0000 7faaa5fa3640 1 -- 172.21.15.47:0/1515228584 <== osd.3 v1:172.21.15.47:6809/32770 1 ==== osd_op_reply(1 rbd_trash [create] v14'1 uv1 ondisk = 0) v8 ==== 153+0+0 (unknown 1742540995 0 0) 0x7faaa0058070 con 0x55c46749ab00 2024-09-15T21:55:18.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.282+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 --> v1:172.21.15.47:6809/32770 -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) v8 -- 0x55c467593870 con 0x55c46749ab00 2024-09-15T21:55:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.282+0000 7faaa5fa3640 1 -- 172.21.15.47:0/1515228584 <== osd.3 v1:172.21.15.47:6809/32770 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 173568156 0 0) 0x7faaa0058070 con 0x55c46749ab00 2024-09-15T21:55:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.282+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 --> v1:172.21.15.47:6809/32770 -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) v8 -- 0x55c467592960 con 0x55c46749ab00 2024-09-15T21:55:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.283+0000 7faaa5fa3640 1 -- 172.21.15.47:0/1515228584 <== osd.3 v1:172.21.15.47:6809/32770 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 185109136 0 0) 0x7faaa0058070 con 0x55c46749ab00 2024-09-15T21:55:18.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:18.283+0000 7faa977fe640 1 -- 172.21.15.47:0/1515228584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) v4 -- 0x7faa80003d30 con 0x55c467483d80 2024-09-15T21:55:19.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:19.277+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 9 ==== pool_op_reply(tid 4 (0) Success v15) v1 ==== 55+0+0 (secure 0 0 0) 0x7faa98041160 con 0x55c467483d80 2024-09-15T21:55:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:19.277+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7faa88051f20 con 0x55c467483d80 2024-09-15T21:55:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:19.278+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 10 ==== osd_map(15..15 src has 1..15) v4 ==== 618+0+0 (secure 0 0 0) 0x7faa9803e080 con 0x55c467483d80 2024-09-15T21:55:19.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:19.278+0000 7faa977fe640 1 -- 172.21.15.47:0/1515228584 --> v1:172.21.15.47:6809/32770 -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x7faa800056d0 con 0x55c46749ab00 2024-09-15T21:55:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:19.281+0000 7faaa5fa3640 1 -- 172.21.15.47:0/1515228584 <== osd.3 v1:172.21.15.47:6809/32770 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v15'1 uv1 ondisk = 0) v8 ==== 194+0+0 (unknown 913822303 0 0) 0x7faaa0058070 con 0x55c46749ab00 2024-09-15T21:55:19.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:19.281+0000 7faa977fe640 1 -- 172.21.15.47:0/1515228584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v15) v4 -- 0x7faa80005140 con 0x55c467483d80 2024-09-15T21:55:20.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.279+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 11 ==== pool_op_reply(tid 6 (0) Success v16) v1 ==== 43+0+0 (secure 0 0 0) 0x7faa98041340 con 0x55c467483d80 2024-09-15T21:55:20.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.279+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=16}) v3 -- 0x7faa880520b0 con 0x55c467483d80 2024-09-15T21:55:20.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.279+0000 7faa957fa640 1 -- 172.21.15.47:0/1515228584 <== mon.0 v2:172.21.15.47:3300/0 12 ==== osd_map(16..16 src has 1..16) v4 ==== 646+0+0 (secure 0 0 0) 0x7faa98061960 con 0x55c467483d80 2024-09-15T21:55:20.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.280+0000 7faa977fe640 1 -- 172.21.15.47:0/1515228584 --> v1:172.21.15.47:6809/32770 -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) v8 -- 0x7faa80005f70 con 0x55c46749ab00 2024-09-15T21:55:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.285+0000 7faaa5fa3640 1 -- 172.21.15.47:0/1515228584 <== osd.3 v1:172.21.15.47:6809/32770 5 ==== osd_op_reply(7 rbd_info [write 0~19] v16'2 uv2 ondisk = 0) v8 ==== 152+0+0 (unknown 2130366450 0 0) 0x7faaa0058070 con 0x55c46749ab00 2024-09-15T21:55:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.286+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 >> v1:172.21.15.47:6809/32770 conn(0x55c46749ab00 legacy=0x55c467544ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.286+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 >> v1:172.21.15.47:6800/32267 conn(0x7faa8804b4a0 legacy=0x7faa8804d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.286+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c467483d80 msgr2=0x55c46752e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.286+0000 7faaa802fb80 1 --2- 172.21.15.47:0/1515228584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c467483d80 0x55c46752e3a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa98008af0 tx=0x7faa9800c040 comp rx=0 tx=0).stop 2024-09-15T21:55:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.286+0000 7faaa5fa3640 1 -- 172.21.15.47:0/1515228584 reap_dead start 2024-09-15T21:55:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.286+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 shutdown_connections 2024-09-15T21:55:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.286+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 >> 172.21.15.47:0/1515228584 conn(0x55c467485e00 msgr2=0x55c4674932b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.286+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 shutdown_connections 2024-09-15T21:55:20.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.286+0000 7faaa802fb80 1 -- 172.21.15.47:0/1515228584 wait complete. 2024-09-15T21:55:20.292 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T21:55:20.293 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T21:55:20.293 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T21:55:20.293 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T21:55:20.294 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T21:55:20.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.435+0000 7ffbf0617640 1 Processor -- start 2024-09-15T21:55:20.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.435+0000 7ffbf0617640 1 -- start start 2024-09-15T21:55:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbf0617640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffbe8095a60 0x7ffbe8095e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbf0617640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe8096370 0x7ffbe812c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbf0617640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe8068df0 con 0x7ffbe812c720 2024-09-15T21:55:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbf0617640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe8090ce0 con 0x7ffbe8095a60 2024-09-15T21:55:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbf0617640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe8090720 con 0x7ffbe8096370 2024-09-15T21:55:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbedbb2640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe8096370 0x7ffbe812c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbee3b3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffbe8095a60 0x7ffbe8095e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbee3b3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffbe8095a60 0x7ffbe8095e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47690/0 (socket says 172.21.15.47:47690) 2024-09-15T21:55:20.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbedbb2640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe8096370 0x7ffbe812c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52204/0 (socket says 172.21.15.47:52204) 2024-09-15T21:55:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbee3b3640 1 -- 172.21.15.47:0/3602634137 learned_addr learned my addr 172.21.15.47:0/3602634137 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbeebb4640 1 --1- 172.21.15.47:0/3602634137 >> v1:172.21.15.47:6789/0 conn(0x7ffbe812c720 0x7ffbe812eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52946/0 (socket says 172.21.15.47:52946) 2024-09-15T21:55:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbee3b3640 1 -- 172.21.15.47:0/3602634137 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe8096370 msgr2=0x7ffbe812c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbee3b3640 1 --2- 172.21.15.47:0/3602634137 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe8096370 0x7ffbe812c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbee3b3640 1 -- 172.21.15.47:0/3602634137 >> v1:172.21.15.47:6789/0 conn(0x7ffbe812c720 legacy=0x7ffbe812eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbed3b1640 1 -- 172.21.15.47:0/3602634137 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026388762 0 0) 0x7ffbe8068df0 con 0x7ffbe812c720 2024-09-15T21:55:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbee3b3640 1 -- 172.21.15.47:0/3602634137 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe8094390 con 0x7ffbe8095a60 2024-09-15T21:55:20.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.436+0000 7ffbedbb2640 1 --2- 172.21.15.47:0/3602634137 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe8096370 0x7ffbe812c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:20.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.437+0000 7ffbee3b3640 1 --2- 172.21.15.47:0/3602634137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbe8095a60 0x7ffbe8095e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009910 tx=0x7ffbe40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4a98ff6fdfe35 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.437+0000 7ffbed3b1640 1 -- 172.21.15.47:0/3602634137 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe402ad40 con 0x7ffbe8095a60 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.437+0000 7ffbed3b1640 1 -- 172.21.15.47:0/3602634137 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe4032440 con 0x7ffbe8095a60 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.437+0000 7ffbed3b1640 1 -- 172.21.15.47:0/3602634137 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe402aea0 con 0x7ffbe8095a60 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.438+0000 7ffbf0617640 1 -- 172.21.15.47:0/3602634137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbe8095a60 msgr2=0x7ffbe8095e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.438+0000 7ffbf0617640 1 --2- 172.21.15.47:0/3602634137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbe8095a60 0x7ffbe8095e30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009910 tx=0x7ffbe40091d0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.438+0000 7ffbf0617640 1 -- 172.21.15.47:0/3602634137 shutdown_connections 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.438+0000 7ffbf0617640 1 --2- 172.21.15.47:0/3602634137 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe8096370 0x7ffbe812c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.438+0000 7ffbf0617640 1 --2- 172.21.15.47:0/3602634137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbe8095a60 0x7ffbe8095e30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.438+0000 7ffbf0617640 1 -- 172.21.15.47:0/3602634137 >> 172.21.15.47:0/3602634137 conn(0x7ffbe809ac50 msgr2=0x7ffbe8096b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.438+0000 7ffbf0617640 1 -- 172.21.15.47:0/3602634137 shutdown_connections 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.438+0000 7ffbf0617640 1 -- 172.21.15.47:0/3602634137 wait complete. 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbf0617640 1 Processor -- start 2024-09-15T21:55:20.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbf0617640 1 -- start start 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbf0617640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffbe8095a60 0x7ffbe808b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbf0617640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbe8096370 0x7ffbe808ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbf0617640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe812c720 0x7ffbe808bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbf0617640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe811fb50 con 0x7ffbe8096370 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbf0617640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe808d470 con 0x7ffbe812c720 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbf0617640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe811f1d0 con 0x7ffbe8095a60 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbee3b3640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffbe8095a60 0x7ffbe808b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbedbb2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbe8096370 0x7ffbe808ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbeebb4640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe812c720 0x7ffbe808bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbee3b3640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffbe8095a60 0x7ffbe808b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48002/0 (socket says 172.21.15.47:48002) 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.439+0000 7ffbee3b3640 1 -- 172.21.15.47:0/1083686060 learned_addr learned my addr 172.21.15.47:0/1083686060 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:20.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbee3b3640 1 -- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe812c720 msgr2=0x7ffbe808bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbee3b3640 1 --2- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe812c720 0x7ffbe808bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbee3b3640 1 -- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbe8096370 msgr2=0x7ffbe808ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbee3b3640 1 --2- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbe8096370 0x7ffbe808ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbee3b3640 1 -- 172.21.15.47:0/1083686060 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8009390 con 0x7ffbe8095a60 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbeebb4640 1 --2- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe812c720 0x7ffbe808bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbedbb2640 1 --2- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbe8096370 0x7ffbe808ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbee3b3640 1 --2- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffbe8095a60 0x7ffbe808b550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffbe402a670 tx=0x7ffbe4032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbde7fc640 1 -- 172.21.15.47:0/1083686060 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4031830 con 0x7ffbe8095a60 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbde7fc640 1 -- 172.21.15.47:0/1083686060 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe4031990 con 0x7ffbe8095a60 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbf0617640 1 -- 172.21.15.47:0/1083686060 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe4009450 con 0x7ffbe8095a60 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbde7fc640 1 -- 172.21.15.47:0/1083686060 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4031d50 con 0x7ffbe8095a60 2024-09-15T21:55:20.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.440+0000 7ffbf0617640 1 -- 172.21.15.47:0/1083686060 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe808d790 con 0x7ffbe8095a60 2024-09-15T21:55:20.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.441+0000 7ffbf0617640 1 -- 172.21.15.47:0/1083686060 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe804bf60 con 0x7ffbe8095a60 2024-09-15T21:55:20.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.441+0000 7ffbde7fc640 1 -- 172.21.15.47:0/1083686060 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7ffbe4039040 con 0x7ffbe8095a60 2024-09-15T21:55:20.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.442+0000 7ffbde7fc640 1 -- 172.21.15.47:0/1083686060 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7ffbe4082f30 con 0x7ffbe8095a60 2024-09-15T21:55:20.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.445+0000 7ffbde7fc640 1 -- 172.21.15.47:0/1083686060 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7ffbe405e070 con 0x7ffbe8095a60 2024-09-15T21:55:20.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.609+0000 7ffbf0617640 1 -- 172.21.15.47:0/1083686060 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ffbe806ecf0 con 0x7ffbe8095a60 2024-09-15T21:55:20.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.611+0000 7ffbde7fc640 1 -- 172.21.15.47:0/1083686060 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+116180 (secure 0 0 0) 0x7ffbe4037070 con 0x7ffbe8095a60 2024-09-15T21:55:20.616 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:55:20.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.613+0000 7ffbf0617640 1 -- 172.21.15.47:0/1083686060 >> v1:172.21.15.47:6800/32267 conn(0x7ffbcc04cb70 legacy=0x7ffbcc04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.613+0000 7ffbf0617640 1 -- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffbe8095a60 msgr2=0x7ffbe808b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.613+0000 7ffbf0617640 1 --2- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffbe8095a60 0x7ffbe808b550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffbe402a670 tx=0x7ffbe4032000 comp rx=0 tx=0).stop 2024-09-15T21:55:20.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.613+0000 7ffbf0617640 1 -- 172.21.15.47:0/1083686060 shutdown_connections 2024-09-15T21:55:20.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.613+0000 7ffbf0617640 1 --2- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffbe812c720 0x7ffbe808bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.613+0000 7ffbf0617640 1 --2- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffbe8096370 0x7ffbe808ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.613+0000 7ffbf0617640 1 --2- 172.21.15.47:0/1083686060 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffbe8095a60 0x7ffbe808b550 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.613+0000 7ffbf0617640 1 -- 172.21.15.47:0/1083686060 >> 172.21.15.47:0/1083686060 conn(0x7ffbe809ac50 msgr2=0x7ffbe8099680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:20.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.613+0000 7ffbf0617640 1 -- 172.21.15.47:0/1083686060 shutdown_connections 2024-09-15T21:55:20.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.613+0000 7ffbf0617640 1 -- 172.21.15.47:0/1083686060 wait complete. 2024-09-15T21:55:20.626 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6800","nonce":32267}]},"active_addr":"172.21.15.47:6800/32267","active_change":"2024-09-15T21:55:02.690184+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4106,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3200219561}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1981014788}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1739125583}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2444109955}]}]} 2024-09-15T21:55:20.628 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T21:55:20.629 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:55:20.629 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:55:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.723+0000 7fcbd9d5d640 1 Processor -- start 2024-09-15T21:55:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd9d5d640 1 -- start start 2024-09-15T21:55:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd9d5d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4096450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd9d5d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcbd412f280 0x7fcbd4131660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd9d5d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd406a070 con 0x7fcbd4096990 2024-09-15T21:55:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd9d5d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd406a720 con 0x7fcbd412f280 2024-09-15T21:55:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd9d5d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd408ef50 con 0x7fcbd4096080 2024-09-15T21:55:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd37fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4096450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcbd412f280 0x7fcbd4131660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd37fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4096450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52224/0 (socket says 172.21.15.47:52224) 2024-09-15T21:55:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd37fe640 1 -- 172.21.15.47:0/179132238 learned_addr learned my addr 172.21.15.47:0/179132238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcbd412f280 0x7fcbd4131660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47716/0 (socket says 172.21.15.47:47716) 2024-09-15T21:55:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd2ffd640 1 --1- 172.21.15.47:0/179132238 >> v1:172.21.15.47:6789/0 conn(0x7fcbd4096990 0x7fcbd4096d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52948/0 (socket says 172.21.15.47:52948) 2024-09-15T21:55:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd37fe640 1 -- 172.21.15.47:0/179132238 >> v2:172.21.15.47:3300/0 conn(0x7fcbd412f280 msgr2=0x7fcbd4131660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd37fe640 1 --2- 172.21.15.47:0/179132238 >> v2:172.21.15.47:3300/0 conn(0x7fcbd412f280 0x7fcbd4131660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd37fe640 1 -- 172.21.15.47:0/179132238 >> v1:172.21.15.47:6789/0 conn(0x7fcbd4096990 legacy=0x7fcbd4096d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd27fc640 1 -- 172.21.15.47:0/179132238 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972881725 0 0) 0x7fcbd406a070 con 0x7fcbd4096990 2024-09-15T21:55:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd37fe640 1 -- 172.21.15.47:0/179132238 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd40699d0 con 0x7fcbd4096080 2024-09-15T21:55:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.724+0000 7fcbd3fff640 1 --2- 172.21.15.47:0/179132238 >> v2:172.21.15.47:3300/0 conn(0x7fcbd412f280 0x7fcbd4131660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.725+0000 7fcbd37fe640 1 --2- 172.21.15.47:0/179132238 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4096450 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009910 tx=0x7fcbc40091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a16b399a8791b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:20.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.725+0000 7fcbd27fc640 1 -- 172.21.15.47:0/179132238 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc402ad20 con 0x7fcbd4096080 2024-09-15T21:55:20.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.725+0000 7fcbd27fc640 1 -- 172.21.15.47:0/179132238 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc402ae80 con 0x7fcbd4096080 2024-09-15T21:55:20.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.725+0000 7fcbd27fc640 1 -- 172.21.15.47:0/179132238 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4032670 con 0x7fcbd4096080 2024-09-15T21:55:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.726+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/179132238 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 msgr2=0x7fcbd4096450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.726+0000 7fcbd9d5d640 1 --2- 172.21.15.47:0/179132238 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4096450 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009910 tx=0x7fcbc40091d0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.726+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/179132238 shutdown_connections 2024-09-15T21:55:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.726+0000 7fcbd9d5d640 1 --2- 172.21.15.47:0/179132238 >> v2:172.21.15.47:3300/0 conn(0x7fcbd412f280 0x7fcbd4131660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.726+0000 7fcbd9d5d640 1 --2- 172.21.15.47:0/179132238 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4096450 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.726+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/179132238 >> 172.21.15.47:0/179132238 conn(0x7fcbd409b450 msgr2=0x7fcbd409b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.726+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/179132238 shutdown_connections 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.726+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/179132238 wait complete. 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd9d5d640 1 Processor -- start 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd9d5d640 1 -- start start 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd9d5d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4139360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd9d5d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcbd4096990 0x7fcbd41398a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd9d5d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd412f280 0x7fcbd4139de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd9d5d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd412e920 con 0x7fcbd412f280 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd9d5d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd412c9b0 con 0x7fcbd4096080 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd9d5d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd412e140 con 0x7fcbd4096990 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd37fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4139360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd412f280 0x7fcbd4139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd2ffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcbd4096990 0x7fcbd41398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd37fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4139360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52234/0 (socket says 172.21.15.47:52234) 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.727+0000 7fcbd37fe640 1 -- 172.21.15.47:0/521939096 learned_addr learned my addr 172.21.15.47:0/521939096 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbd3fff640 1 -- 172.21.15.47:0/521939096 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcbd4096990 msgr2=0x7fcbd41398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbd3fff640 1 --2- 172.21.15.47:0/521939096 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcbd4096990 0x7fcbd41398a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbd3fff640 1 -- 172.21.15.47:0/521939096 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 msgr2=0x7fcbd4139360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbd3fff640 1 --2- 172.21.15.47:0/521939096 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4139360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbd3fff640 1 -- 172.21.15.47:0/521939096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc009a20 con 0x7fcbd412f280 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbd3fff640 1 --2- 172.21.15.47:0/521939096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd412f280 0x7fcbd4139de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009ee0 tx=0x7fcbcc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbbb7fe640 1 -- 172.21.15.47:0/521939096 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00ce10 con 0x7fcbd412f280 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbbb7fe640 1 -- 172.21.15.47:0/521939096 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc00f040 con 0x7fcbd412f280 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/521939096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc4009450 con 0x7fcbd412f280 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbbb7fe640 1 -- 172.21.15.47:0/521939096 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0146d0 con 0x7fcbd412f280 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.728+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/521939096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd412b690 con 0x7fcbd412f280 2024-09-15T21:55:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.729+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/521939096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd404bf60 con 0x7fcbd412f280 2024-09-15T21:55:20.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.729+0000 7fcbbb7fe640 1 -- 172.21.15.47:0/521939096 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fcbcc003020 con 0x7fcbd412f280 2024-09-15T21:55:20.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.730+0000 7fcbbb7fe640 1 -- 172.21.15.47:0/521939096 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fcbcc063b30 con 0x7fcbd412f280 2024-09-15T21:55:20.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.733+0000 7fcbbb7fe640 1 -- 172.21.15.47:0/521939096 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fcbcc03ec70 con 0x7fcbd412f280 2024-09-15T21:55:20.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.870+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/521939096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbd40646b0 con 0x7fcbd412f280 2024-09-15T21:55:20.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.871+0000 7fcbbb7fe640 1 -- 172.21.15.47:0/521939096 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8169 (secure 0 0 0) 0x7fcbcc018030 con 0x7fcbd412f280 2024-09-15T21:55:20.873 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:55:20.874 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":16,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:55:20.276874+0000","last_up_change":"2024-09-15T21:55:10.248544+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":32760}]},"public_addr":"172.21.15.47:6801/32760","cluster_addr":"172.21.15.47:6802/32760","heartbeat_back_addr":"172.21.15.47:6804/32760","heartbeat_front_addr":"172.21.15.47:6803/32760","state":["exists","up"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":32761}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":32761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":32761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":32761}]},"public_addr":"172.21.15.47:6805/32761","cluster_addr":"172.21.15.47:6806/32761","heartbeat_back_addr":"172.21.15.47:6808/32761","heartbeat_front_addr":"172.21.15.47:6807/32761","state":["exists","up"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":32772}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":32772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":32772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":32772}]},"public_addr":"172.21.15.47:6813/32772","cluster_addr":"172.21.15.47:6814/32772","heartbeat_back_addr":"172.21.15.47:6816/32772","heartbeat_front_addr":"172.21.15.47:6815/32772","state":["exists","up"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":32770}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":32770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":32770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":32770}]},"public_addr":"172.21.15.47:6809/32770","cluster_addr":"172.21.15.47:6810/32770","heartbeat_back_addr":"172.21.15.47:6812/32770","heartbeat_front_addr":"172.21.15.47:6811/32770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:55:20.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.872+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/521939096 >> v1:172.21.15.47:6800/32267 conn(0x7fcbc004cb70 legacy=0x7fcbc004f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.872+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/521939096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd412f280 msgr2=0x7fcbd4139de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.872+0000 7fcbd9d5d640 1 --2- 172.21.15.47:0/521939096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd412f280 0x7fcbd4139de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009ee0 tx=0x7fcbcc00b040 comp rx=0 tx=0).stop 2024-09-15T21:55:20.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.872+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/521939096 shutdown_connections 2024-09-15T21:55:20.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.872+0000 7fcbd9d5d640 1 --2- 172.21.15.47:0/521939096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcbd412f280 0x7fcbd4139de0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.872+0000 7fcbd9d5d640 1 --2- 172.21.15.47:0/521939096 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fcbd4096990 0x7fcbd41398a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.872+0000 7fcbd9d5d640 1 --2- 172.21.15.47:0/521939096 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fcbd4096080 0x7fcbd4139360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.872+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/521939096 >> 172.21.15.47:0/521939096 conn(0x7fcbd409b450 msgr2=0x7fcbd408d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:20.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.872+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/521939096 shutdown_connections 2024-09-15T21:55:20.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.872+0000 7fcbd9d5d640 1 -- 172.21.15.47:0/521939096 wait complete. 2024-09-15T21:55:20.885 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:55:20.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:55:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f95370dd640 1 Processor -- start 2024-09-15T21:55:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f95370dd640 1 -- start start 2024-09-15T21:55:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f95370dd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9530094980 0x7f95301270e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f95370dd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127a60 0x7f9530091c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f95370dd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95300933f0 con 0x7f9530127620 2024-09-15T21:55:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f95370dd640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9530095020 con 0x7f9530094980 2024-09-15T21:55:20.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f95370dd640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9530092660 con 0x7f9530127a60 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f953567a640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127a60 0x7f9530091c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f9534e79640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9530094980 0x7f95301270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f953567a640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127a60 0x7f9530091c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48016/0 (socket says 172.21.15.47:48016) 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f9534e79640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9530094980 0x7f95301270e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47740/0 (socket says 172.21.15.47:47740) 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f953567a640 1 -- 172.21.15.47:0/2444328939 learned_addr learned my addr 172.21.15.47:0/2444328939 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.979+0000 7f952ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9530127620 0x7f9530091570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52954/0 (socket says 172.21.15.47:52954) 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.980+0000 7f9534e79640 1 -- 172.21.15.47:0/2444328939 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127a60 msgr2=0x7f9530091c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.980+0000 7f9534e79640 1 --2- 172.21.15.47:0/2444328939 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127a60 0x7f9530091c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.980+0000 7f9534e79640 1 -- 172.21.15.47:0/2444328939 >> v1:172.21.15.47:6789/0 conn(0x7f9530127620 legacy=0x7f9530091570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.980+0000 7f952f7fe640 1 -- 172.21.15.47:0/2444328939 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394692747 0 0) 0x7f95300933f0 con 0x7f9530127620 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.980+0000 7f9534e79640 1 -- 172.21.15.47:0/2444328939 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953006e590 con 0x7f9530094980 2024-09-15T21:55:20.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.980+0000 7f953567a640 1 --2- 172.21.15.47:0/2444328939 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127a60 0x7f9530091c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:20.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f9534e79640 1 --2- 172.21.15.47:0/2444328939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9530094980 0x7f95301270e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9524009850 tx=0x7f95240090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d51ae9305f271b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:20.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f952f7fe640 1 -- 172.21.15.47:0/2444328939 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952402ad00 con 0x7f9530094980 2024-09-15T21:55:20.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f952f7fe640 1 -- 172.21.15.47:0/2444328939 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952402ae60 con 0x7f9530094980 2024-09-15T21:55:20.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f952f7fe640 1 -- 172.21.15.47:0/2444328939 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524032670 con 0x7f9530094980 2024-09-15T21:55:20.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f95370dd640 1 -- 172.21.15.47:0/2444328939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9530094980 msgr2=0x7f95301270e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f95370dd640 1 --2- 172.21.15.47:0/2444328939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9530094980 0x7f95301270e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9524009850 tx=0x7f95240090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f95370dd640 1 -- 172.21.15.47:0/2444328939 shutdown_connections 2024-09-15T21:55:20.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f95370dd640 1 --2- 172.21.15.47:0/2444328939 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127a60 0x7f9530091c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f95370dd640 1 --2- 172.21.15.47:0/2444328939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9530094980 0x7f95301270e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f95370dd640 1 -- 172.21.15.47:0/2444328939 >> 172.21.15.47:0/2444328939 conn(0x7f9530095a70 msgr2=0x7f9530095e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f95370dd640 1 -- 172.21.15.47:0/2444328939 shutdown_connections 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.981+0000 7f95370dd640 1 -- 172.21.15.47:0/2444328939 wait complete. 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.982+0000 7f95370dd640 1 Processor -- start 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.982+0000 7f95370dd640 1 -- start start 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f95370dd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9530094980 0x7f9530137410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f95370dd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127620 0x7f9530137950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f95370dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9530127a60 0x7f953008de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f95370dd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953011f1b0 con 0x7f9530127a60 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f95370dd640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95300986c0 con 0x7f9530094980 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f95370dd640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f953011f4c0 con 0x7f9530127620 2024-09-15T21:55:20.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f9534e79640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9530094980 0x7f9530137410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f952ffff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127620 0x7f9530137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f9534e79640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9530094980 0x7f9530137410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52248/0 (socket says 172.21.15.47:52248) 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f9534e79640 1 -- 172.21.15.47:0/2785735304 learned_addr learned my addr 172.21.15.47:0/2785735304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f953567a640 1 --2- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9530127a60 0x7f953008de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f9534e79640 1 -- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127620 msgr2=0x7f9530137950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f9534e79640 1 --2- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127620 0x7f9530137950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f9534e79640 1 -- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9530127a60 msgr2=0x7f953008de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f9534e79640 1 --2- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9530127a60 0x7f953008de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f9534e79640 1 -- 172.21.15.47:0/2785735304 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9528009ae0 con 0x7f9530094980 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f953567a640 1 --2- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9530127a60 0x7f953008de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.983+0000 7f9534e79640 1 --2- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9530094980 0x7f9530137410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f952402a630 tx=0x7f9524032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.984+0000 7f952cff9640 1 -- 172.21.15.47:0/2785735304 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524031700 con 0x7f9530094980 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.984+0000 7f952cff9640 1 -- 172.21.15.47:0/2785735304 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9524031860 con 0x7f9530094980 2024-09-15T21:55:20.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.984+0000 7f952cff9640 1 -- 172.21.15.47:0/2785735304 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524031c20 con 0x7f9530094980 2024-09-15T21:55:20.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.984+0000 7f95370dd640 1 -- 172.21.15.47:0/2785735304 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9524009390 con 0x7f9530094980 2024-09-15T21:55:20.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.984+0000 7f95370dd640 1 -- 172.21.15.47:0/2785735304 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9530137e90 con 0x7f9530094980 2024-09-15T21:55:20.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.985+0000 7f95370dd640 1 -- 172.21.15.47:0/2785735304 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953008f8e0 con 0x7f9530094980 2024-09-15T21:55:20.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.985+0000 7f952cff9640 1 -- 172.21.15.47:0/2785735304 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f9524039040 con 0x7f9530094980 2024-09-15T21:55:20.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.985+0000 7f952cff9640 1 -- 172.21.15.47:0/2785735304 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f9524083470 con 0x7f9530094980 2024-09-15T21:55:20.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:20.989+0000 7f952cff9640 1 -- 172.21.15.47:0/2785735304 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f952405e4c0 con 0x7f9530094980 2024-09-15T21:55:21.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.126+0000 7f95370dd640 1 -- 172.21.15.47:0/2785735304 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f953004bf60 con 0x7f9530094980 2024-09-15T21:55:21.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.127+0000 7f952cff9640 1 -- 172.21.15.47:0/2785735304 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8169 (secure 0 0 0) 0x7f952403f070 con 0x7f9530094980 2024-09-15T21:55:21.129 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:55:21.129 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":16,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:55:20.276874+0000","last_up_change":"2024-09-15T21:55:10.248544+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":32760}]},"public_addr":"172.21.15.47:6801/32760","cluster_addr":"172.21.15.47:6802/32760","heartbeat_back_addr":"172.21.15.47:6804/32760","heartbeat_front_addr":"172.21.15.47:6803/32760","state":["exists","up"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":32761}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":32761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":32761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":32761}]},"public_addr":"172.21.15.47:6805/32761","cluster_addr":"172.21.15.47:6806/32761","heartbeat_back_addr":"172.21.15.47:6808/32761","heartbeat_front_addr":"172.21.15.47:6807/32761","state":["exists","up"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":32772}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":32772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":32772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":32772}]},"public_addr":"172.21.15.47:6813/32772","cluster_addr":"172.21.15.47:6814/32772","heartbeat_back_addr":"172.21.15.47:6816/32772","heartbeat_front_addr":"172.21.15.47:6815/32772","state":["exists","up"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":32770}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":32770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":32770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":32770}]},"public_addr":"172.21.15.47:6809/32770","cluster_addr":"172.21.15.47:6810/32770","heartbeat_back_addr":"172.21.15.47:6812/32770","heartbeat_front_addr":"172.21.15.47:6811/32770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:55:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.129+0000 7f95370dd640 1 -- 172.21.15.47:0/2785735304 >> v1:172.21.15.47:6800/32267 conn(0x7f951404cb70 legacy=0x7f951404f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.129+0000 7f95370dd640 1 -- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9530094980 msgr2=0x7f9530137410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.129+0000 7f95370dd640 1 --2- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9530094980 0x7f9530137410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f952402a630 tx=0x7f9524032000 comp rx=0 tx=0).stop 2024-09-15T21:55:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.129+0000 7f95370dd640 1 -- 172.21.15.47:0/2785735304 shutdown_connections 2024-09-15T21:55:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.129+0000 7f95370dd640 1 --2- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9530127a60 0x7f953008de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.129+0000 7f95370dd640 1 --2- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9530127620 0x7f9530137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.129+0000 7f95370dd640 1 --2- 172.21.15.47:0/2785735304 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9530094980 0x7f9530137410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.129+0000 7f95370dd640 1 -- 172.21.15.47:0/2785735304 >> 172.21.15.47:0/2785735304 conn(0x7f9530095a70 msgr2=0x7f953006d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.129+0000 7f95370dd640 1 -- 172.21.15.47:0/2785735304 shutdown_connections 2024-09-15T21:55:21.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.129+0000 7f95370dd640 1 -- 172.21.15.47:0/2785735304 wait complete. 2024-09-15T21:55:21.142 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:55:21.142 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:55:21.142 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:55:21.143 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T21:55:21.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7fa660d5d640 1 Processor -- start 2024-09-15T21:55:21.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7fa660d5d640 1 -- start start 2024-09-15T21:55:21.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7f933219d640 1 Processor -- start 2024-09-15T21:55:21.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7fdf45d00640 1 Processor -- start 2024-09-15T21:55:21.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7f1e42f5d640 1 Processor -- start 2024-09-15T21:55:21.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7f933219d640 1 -- start start 2024-09-15T21:55:21.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7fdf45d00640 1 -- start start 2024-09-15T21:55:21.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7f1e42f5d640 1 -- start start 2024-09-15T21:55:21.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7f1e42f5d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1e3c090460 0x7f1e3c090830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7f1e42f5d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e3c090d70 0x7f1e3c12c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7f1e42f5d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e3c12c650 0x7f1e3c12ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7f1e42f5d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c095900 con 0x7f1e3c090460 2024-09-15T21:55:21.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7f1e42f5d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0914e0 con 0x7f1e3c12c650 2024-09-15T21:55:21.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.243+0000 7f1e42f5d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0922e0 con 0x7f1e3c090d70 2024-09-15T21:55:21.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fa660d5d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c129c70 0x7fa65c12a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fa660d5d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa65c12a580 0x7fa65c0969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fdf45d00640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf40090f40 0x7fdf4012d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fdf45d00640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf4012d660 0x7fdf4012fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fa660d5d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c0998b0 con 0x7fa65c096f20 2024-09-15T21:55:21.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fa65a59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c129c70 0x7fa65c12a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f933219d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f932c096540 0x7f932c12f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f933219d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c12f750 0x7f932c131b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f933219d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c0689e0 con 0x7f932c095a60 2024-09-15T21:55:21.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f933219d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c068140 con 0x7f932c096540 2024-09-15T21:55:21.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f933219d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c069010 con 0x7f932c12f750 2024-09-15T21:55:21.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fa65a59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c129c70 0x7fa65c12a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52264/0 (socket says 172.21.15.47:52264) 2024-09-15T21:55:21.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fa65a59c640 1 -- 172.21.15.47:0/3275779118 learned_addr learned my addr 172.21.15.47:0/3275779118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fa660d5d640 1 -- 172.21.15.47:0/3275779118 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c09a180 con 0x7fa65c12a580 2024-09-15T21:55:21.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fdf45d00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf400922e0 con 0x7fdf40090460 2024-09-15T21:55:21.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fdf45d00640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40095900 con 0x7fdf40090f40 2024-09-15T21:55:21.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fa660d5d640 1 -- 172.21.15.47:0/3275779118 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c09a440 con 0x7fa65c129c70 2024-09-15T21:55:21.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fdf45d00640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400914e0 con 0x7fdf4012d660 2024-09-15T21:55:21.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f932b7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f932c095a60 0x7f932c095e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52972/0 (socket says 172.21.15.47:52972) 2024-09-15T21:55:21.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f932b7fe640 1 -- 172.21.15.47:0/559401714 learned_addr learned my addr 172.21.15.47:0/559401714 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:21.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f932affd640 1 --2- 172.21.15.47:0/559401714 >> v2:172.21.15.47:3300/0 conn(0x7f932c096540 0x7f932c12f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fdf3effd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf40090f40 0x7fdf4012d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f1e40cf9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1e3c090460 0x7f1e3c090830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f1e3bfff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e3c090d70 0x7f1e3c12c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fa659d9b640 1 --2- 172.21.15.47:0/3275779118 >> v2:172.21.15.47:3300/0 conn(0x7fa65c12a580 0x7fa65c0969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f1e40cf9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1e3c090460 0x7f1e3c090830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47756/0 (socket says 172.21.15.47:47756) 2024-09-15T21:55:21.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f1e40cf9640 1 -- 172.21.15.47:0/2871844922 learned_addr learned my addr 172.21.15.47:0/2871844922 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f1e414fa640 1 --2- 172.21.15.47:0/2871844922 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e3c12c650 0x7f1e3c12ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fdf3ffff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf4012d660 0x7fdf4012fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fdf3ffff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf4012d660 0x7fdf4012fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48038/0 (socket says 172.21.15.47:48038) 2024-09-15T21:55:21.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fdf3ffff640 1 -- 172.21.15.47:0/3115632250 learned_addr learned my addr 172.21.15.47:0/3115632250 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f932bfff640 1 --2- 172.21.15.47:0/559401714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c12f750 0x7f932c131b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7fdf3effd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf40090f40 0x7fdf4012d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52274/0 (socket says 172.21.15.47:52274) 2024-09-15T21:55:21.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7fdf3ffff640 1 -- 172.21.15.47:0/3115632250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf40090f40 msgr2=0x7fdf4012d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7fdf3ffff640 1 --2- 172.21.15.47:0/3115632250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf40090f40 0x7fdf4012d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7fdf3ffff640 1 -- 172.21.15.47:0/3115632250 >> v1:172.21.15.47:6789/0 conn(0x7fdf40090460 legacy=0x7fdf40090830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7fdf3ffff640 1 -- 172.21.15.47:0/3115632250 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf40095260 con 0x7fdf4012d660 2024-09-15T21:55:21.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f1e3bfff640 1 -- 172.21.15.47:0/2871844922 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e3c12c650 msgr2=0x7f1e3c12ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f1e3bfff640 1 --2- 172.21.15.47:0/2871844922 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e3c12c650 0x7f1e3c12ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f1e3bfff640 1 -- 172.21.15.47:0/2871844922 >> v2:172.21.15.47:3300/0 conn(0x7f1e3c090460 msgr2=0x7f1e3c090830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f1e3bfff640 1 --2- 172.21.15.47:0/2871844922 >> v2:172.21.15.47:3300/0 conn(0x7f1e3c090460 0x7f1e3c090830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.244+0000 7f1e3bfff640 1 -- 172.21.15.47:0/2871844922 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c095260 con 0x7f1e3c090d70 2024-09-15T21:55:21.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7f932affd640 1 -- 172.21.15.47:0/559401714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c12f750 msgr2=0x7f932c131b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7f932affd640 1 --2- 172.21.15.47:0/559401714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c12f750 0x7f932c131b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7f932affd640 1 -- 172.21.15.47:0/559401714 >> v1:172.21.15.47:6789/0 conn(0x7f932c095a60 legacy=0x7f932c095e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7fa659d9b640 1 -- 172.21.15.47:0/3275779118 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c129c70 msgr2=0x7fa65c12a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7fa659d9b640 1 --2- 172.21.15.47:0/3275779118 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c129c70 0x7fa65c12a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7fa659d9b640 1 -- 172.21.15.47:0/3275779118 >> v1:172.21.15.47:6789/0 conn(0x7fa65c096f20 legacy=0x7fa65c0973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7fa659d9b640 1 -- 172.21.15.47:0/3275779118 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c099db0 con 0x7fa65c12a580 2024-09-15T21:55:21.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7f932affd640 1 -- 172.21.15.47:0/559401714 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c06ec40 con 0x7f932c096540 2024-09-15T21:55:21.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.245+0000 7f1e3bfff640 1 --2- 172.21.15.47:0/2871844922 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e3c090d70 0x7f1e3c12c110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c013f00 tx=0x7f1e2c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79fd18df46e5547b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7f1e3b7fe640 1 -- 172.21.15.47:0/2871844922 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c037510 con 0x7f1e3c090d70 2024-09-15T21:55:21.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7f1e3b7fe640 1 -- 172.21.15.47:0/2871844922 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c035050 con 0x7f1e3c090d70 2024-09-15T21:55:21.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7f1e3b7fe640 1 -- 172.21.15.47:0/2871844922 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0378e0 con 0x7f1e3c090d70 2024-09-15T21:55:21.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7fdf3ffff640 1 --2- 172.21.15.47:0/3115632250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf4012d660 0x7fdf4012fa40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009f50 tx=0x7fdf38009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a69511de72bbec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7fa659d9b640 1 --2- 172.21.15.47:0/3275779118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c12a580 0x7fa65c0969e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa650009850 tx=0x7fa6500090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9373b3b6d800c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7fdf3e7fc640 1 -- 172.21.15.47:0/3115632250 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38031400 con 0x7fdf4012d660 2024-09-15T21:55:21.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7fdf3e7fc640 1 -- 172.21.15.47:0/3115632250 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3802d050 con 0x7fdf4012d660 2024-09-15T21:55:21.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7f932affd640 1 --2- 172.21.15.47:0/559401714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f932c096540 0x7f932c12f210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9320009880 tx=0x7f93200090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b03fc0bd922636d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7fdf3e7fc640 1 -- 172.21.15.47:0/3115632250 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38031790 con 0x7fdf4012d660 2024-09-15T21:55:21.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7f932a7fc640 1 -- 172.21.15.47:0/559401714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932002ac50 con 0x7f932c096540 2024-09-15T21:55:21.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7fa65959a640 1 -- 172.21.15.47:0/3275779118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65002ac60 con 0x7fa65c12a580 2024-09-15T21:55:21.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f1e42f5d640 1 -- 172.21.15.47:0/2871844922 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e3c090d70 msgr2=0x7f1e3c12c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f1e42f5d640 1 --2- 172.21.15.47:0/2871844922 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e3c090d70 0x7f1e3c12c110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c013f00 tx=0x7f1e2c0090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f1e42f5d640 1 -- 172.21.15.47:0/2871844922 shutdown_connections 2024-09-15T21:55:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7f932a7fc640 1 -- 172.21.15.47:0/559401714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932002adb0 con 0x7f932c096540 2024-09-15T21:55:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7f932a7fc640 1 -- 172.21.15.47:0/559401714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320032670 con 0x7f932c096540 2024-09-15T21:55:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7fa65959a640 1 -- 172.21.15.47:0/3275779118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65002adc0 con 0x7fa65c12a580 2024-09-15T21:55:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.246+0000 7fa65959a640 1 -- 172.21.15.47:0/3275779118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650032670 con 0x7fa65c12a580 2024-09-15T21:55:21.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fa660d5d640 1 -- 172.21.15.47:0/3275779118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c12a580 msgr2=0x7fa65c0969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fa660d5d640 1 --2- 172.21.15.47:0/3275779118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c12a580 0x7fa65c0969e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa650009850 tx=0x7fa6500090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f1e42f5d640 1 --2- 172.21.15.47:0/2871844922 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e3c12c650 0x7f1e3c12ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f1e42f5d640 1 --2- 172.21.15.47:0/2871844922 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e3c090d70 0x7f1e3c12c110 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fdf45d00640 1 -- 172.21.15.47:0/3115632250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf4012d660 msgr2=0x7fdf4012fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fdf45d00640 1 --2- 172.21.15.47:0/3115632250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf4012d660 0x7fdf4012fa40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009f50 tx=0x7fdf38009840 comp rx=0 tx=0).stop 2024-09-15T21:55:21.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fdf45d00640 1 -- 172.21.15.47:0/3115632250 shutdown_connections 2024-09-15T21:55:21.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fdf45d00640 1 --2- 172.21.15.47:0/3115632250 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf4012d660 0x7fdf4012fa40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fdf45d00640 1 --2- 172.21.15.47:0/3115632250 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf40090f40 0x7fdf4012d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fdf45d00640 1 -- 172.21.15.47:0/3115632250 >> 172.21.15.47:0/3115632250 conn(0x7fdf40095a70 msgr2=0x7fdf40095e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fa660d5d640 1 -- 172.21.15.47:0/3275779118 shutdown_connections 2024-09-15T21:55:21.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fa660d5d640 1 --2- 172.21.15.47:0/3275779118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c12a580 0x7fa65c0969e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fa660d5d640 1 --2- 172.21.15.47:0/3275779118 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c129c70 0x7fa65c12a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fa660d5d640 1 -- 172.21.15.47:0/3275779118 >> 172.21.15.47:0/3275779118 conn(0x7fa65c09d850 msgr2=0x7fa65c09dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fdf45d00640 1 -- 172.21.15.47:0/3115632250 shutdown_connections 2024-09-15T21:55:21.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f1e42f5d640 1 --2- 172.21.15.47:0/2871844922 >> v2:172.21.15.47:3300/0 conn(0x7f1e3c090460 0x7f1e3c090830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f1e42f5d640 1 -- 172.21.15.47:0/2871844922 >> 172.21.15.47:0/2871844922 conn(0x7f1e3c095a70 msgr2=0x7f1e3c095e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fdf45d00640 1 -- 172.21.15.47:0/3115632250 wait complete. 2024-09-15T21:55:21.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7fa660d5d640 1 -- 172.21.15.47:0/3275779118 shutdown_connections 2024-09-15T21:55:21.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f933219d640 1 -- 172.21.15.47:0/559401714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f932c096540 msgr2=0x7f932c12f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f933219d640 1 --2- 172.21.15.47:0/559401714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f932c096540 0x7f932c12f210 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9320009880 tx=0x7f93200090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f1e42f5d640 1 -- 172.21.15.47:0/2871844922 shutdown_connections 2024-09-15T21:55:21.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f933219d640 1 -- 172.21.15.47:0/559401714 shutdown_connections 2024-09-15T21:55:21.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f933219d640 1 --2- 172.21.15.47:0/559401714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c12f750 0x7f932c131b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f933219d640 1 --2- 172.21.15.47:0/559401714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f932c096540 0x7f932c12f210 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f933219d640 1 -- 172.21.15.47:0/559401714 >> 172.21.15.47:0/559401714 conn(0x7f932c09ac50 msgr2=0x7f932c096b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f933219d640 1 -- 172.21.15.47:0/559401714 shutdown_connections 2024-09-15T21:55:21.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f1e42f5d640 1 -- 172.21.15.47:0/2871844922 wait complete. 2024-09-15T21:55:21.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.247+0000 7f933219d640 1 -- 172.21.15.47:0/559401714 wait complete. 2024-09-15T21:55:21.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.248+0000 7fa660d5d640 1 -- 172.21.15.47:0/3275779118 wait complete. 2024-09-15T21:55:21.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.248+0000 7fa660d5d640 1 Processor -- start 2024-09-15T21:55:21.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.248+0000 7fdf45d00640 1 Processor -- start 2024-09-15T21:55:21.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.248+0000 7fa660d5d640 1 -- start start 2024-09-15T21:55:21.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf45d00640 1 -- start start 2024-09-15T21:55:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7f933219d640 1 Processor -- start 2024-09-15T21:55:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7f1e42f5d640 1 Processor -- start 2024-09-15T21:55:21.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7f933219d640 1 -- start start 2024-09-15T21:55:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa660d5d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c096f20 0x7fa65c119c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa660d5d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa65c129c70 0x7fa65c11a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa660d5d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c12a580 0x7fa65c11a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa660d5d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c11ca00 con 0x7fa65c12a580 2024-09-15T21:55:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa660d5d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c11b7b0 con 0x7fa65c096f20 2024-09-15T21:55:21.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf45d00640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf40090460 0x7fdf4008e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf45d00640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf40090f40 0x7fdf4008ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa660d5d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c11acf0 con 0x7fa65c129c70 2024-09-15T21:55:21.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf45d00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf4012d660 0x7fdf4008f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf45d00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4006b390 con 0x7fdf4012d660 2024-09-15T21:55:21.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7f1e42f5d640 1 -- start start 2024-09-15T21:55:21.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf45d00640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4012c5b0 con 0x7fdf40090f40 2024-09-15T21:55:21.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf45d00640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4006af50 con 0x7fdf40090460 2024-09-15T21:55:21.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa659d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa65c129c70 0x7fa65c11a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa65a59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c096f20 0x7fa65c119c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa65ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c12a580 0x7fa65c11a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa65ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c12a580 0x7fa65c11a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47776/0 (socket says 172.21.15.47:47776) 2024-09-15T21:55:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7f933219d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c095a60 0x7f932c08b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7f933219d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f932c12f750 0x7f932c08bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7f933219d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f932c08c300 0x7f932c1395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa65ad9d640 1 -- 172.21.15.47:0/730062048 learned_addr learned my addr 172.21.15.47:0/730062048 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7f933219d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c1201c0 con 0x7f932c12f750 2024-09-15T21:55:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7f933219d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c08d460 con 0x7f932c095a60 2024-09-15T21:55:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7f933219d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f932c08c930 con 0x7f932c08c300 2024-09-15T21:55:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf3ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf4012d660 0x7fdf4008f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf3ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf4012d660 0x7fdf4008f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47790/0 (socket says 172.21.15.47:47790) 2024-09-15T21:55:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf3ffff640 1 -- 172.21.15.47:0/3797624008 learned_addr learned my addr 172.21.15.47:0/3797624008 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf3effd640 1 --2- 172.21.15.47:0/3797624008 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf40090f40 0x7fdf4008ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e42f5d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e3c090460 0x7f1e3c08c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e42f5d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e3c090d70 0x7f1e3c08ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fdf3f7fe640 1 --2- 172.21.15.47:0/3797624008 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf40090460 0x7fdf4008e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e42f5d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e3c12c650 0x7f1e3c08d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e42f5d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c08f9a0 con 0x7f1e3c090460 2024-09-15T21:55:21.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e42f5d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c08fd30 con 0x7f1e3c090d70 2024-09-15T21:55:21.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e42f5d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c12b570 con 0x7f1e3c12c650 2024-09-15T21:55:21.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fdf3ffff640 1 -- 172.21.15.47:0/3797624008 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf40090460 msgr2=0x7fdf4008e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fdf3ffff640 1 --2- 172.21.15.47:0/3797624008 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdf40090460 0x7fdf4008e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fdf3ffff640 1 -- 172.21.15.47:0/3797624008 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf40090f40 msgr2=0x7fdf4008ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fdf3ffff640 1 --2- 172.21.15.47:0/3797624008 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdf40090f40 0x7fdf4008ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fdf3ffff640 1 -- 172.21.15.47:0/3797624008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf38009ae0 con 0x7fdf4012d660 2024-09-15T21:55:21.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e40cf9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e3c090460 0x7f1e3c08c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e40cf9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e3c090460 0x7f1e3c08c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47816/0 (socket says 172.21.15.47:47816) 2024-09-15T21:55:21.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f932b7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c095a60 0x7f932c08b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f932b7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c095a60 0x7f932c08b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52304/0 (socket says 172.21.15.47:52304) 2024-09-15T21:55:21.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f932b7fe640 1 -- 172.21.15.47:0/3265366229 learned_addr learned my addr 172.21.15.47:0/3265366229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e40cf9640 1 -- 172.21.15.47:0/3300977634 learned_addr learned my addr 172.21.15.47:0/3300977634 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e3bfff640 1 --2- 172.21.15.47:0/3300977634 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e3c090d70 0x7f1e3c08ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fdf3ffff640 1 --2- 172.21.15.47:0/3797624008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf4012d660 0x7fdf4008f270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009f20 tx=0x7fdf38036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f932affd640 1 --2- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f932c12f750 0x7f932c08bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e40cf9640 1 -- 172.21.15.47:0/3300977634 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e3c12c650 msgr2=0x7f1e3c08d3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:55:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e40cf9640 1 --2- 172.21.15.47:0/3300977634 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1e3c12c650 0x7f1e3c08d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fdf277fe640 1 -- 172.21.15.47:0/3797624008 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf380077e0 con 0x7fdf4012d660 2024-09-15T21:55:21.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7f1e40cf9640 1 -- 172.21.15.47:0/3300977634 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e3c090d70 msgr2=0x7f1e3c08ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fdf277fe640 1 -- 172.21.15.47:0/3797624008 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf38007940 con 0x7fdf4012d660 2024-09-15T21:55:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4006ebf0 con 0x7fdf4012d660 2024-09-15T21:55:21.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f932b7fe640 1 -- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f932c08c300 msgr2=0x7f932c1395f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.249+0000 7fa659d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa65c129c70 0x7fa65c11a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48046/0 (socket says 172.21.15.47:48046) 2024-09-15T21:55:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fa659d9b640 1 -- 172.21.15.47:0/730062048 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c096f20 msgr2=0x7fa65c119c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fa659d9b640 1 --2- 172.21.15.47:0/730062048 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c096f20 0x7fa65c119c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fa659d9b640 1 -- 172.21.15.47:0/730062048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c12a580 msgr2=0x7fa65c11a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fa659d9b640 1 --2- 172.21.15.47:0/730062048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c12a580 0x7fa65c11a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fa659d9b640 1 -- 172.21.15.47:0/730062048 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa650009390 con 0x7fa65c129c70 2024-09-15T21:55:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.250+0000 7fa65ad9d640 1 --2- 172.21.15.47:0/730062048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa65c12a580 0x7fa65c11a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7fa65a59c640 1 --2- 172.21.15.47:0/730062048 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa65c096f20 0x7fa65c119c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:55:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f932bfff640 1 --2- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f932c08c300 0x7f932c1395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f932b7fe640 1 --2- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f932c08c300 0x7f932c1395f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f932b7fe640 1 -- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f932c12f750 msgr2=0x7f932c08bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f932b7fe640 1 --2- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f932c12f750 0x7f932c08bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f932b7fe640 1 -- 172.21.15.47:0/3265366229 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9320009390 con 0x7f932c095a60 2024-09-15T21:55:21.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f932affd640 1 --2- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f932c12f750 0x7f932c08bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:21.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7fdf277fe640 1 -- 172.21.15.47:0/3797624008 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38007d00 con 0x7fdf4012d660 2024-09-15T21:55:21.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f1e40cf9640 1 --2- 172.21.15.47:0/3300977634 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e3c090d70 0x7f1e3c08ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f1e40cf9640 1 -- 172.21.15.47:0/3300977634 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c013a10 con 0x7f1e3c090460 2024-09-15T21:55:21.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f932bfff640 1 --2- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f932c08c300 0x7f932c1395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:55:21.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f1e3bfff640 1 --2- 172.21.15.47:0/3300977634 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1e3c090d70 0x7f1e3c08ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:21.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4012cfe0 con 0x7fdf4012d660 2024-09-15T21:55:21.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7fa659d9b640 1 --2- 172.21.15.47:0/730062048 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa65c129c70 0x7fa65c11a180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa650031000 tx=0x7fa650032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f1e40cf9640 1 --2- 172.21.15.47:0/3300977634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e3c090460 0x7f1e3c08c940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000ea50 tx=0x7f1e300091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f932b7fe640 1 --2- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c095a60 0x7f932c08b880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c590 tx=0x7f931c007ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7fa64a7fc640 1 -- 172.21.15.47:0/730062048 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650041cb0 con 0x7fa65c129c70 2024-09-15T21:55:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f93137fe640 1 -- 172.21.15.47:0/3265366229 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c005aa0 con 0x7f932c095a60 2024-09-15T21:55:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7fa64a7fc640 1 -- 172.21.15.47:0/730062048 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa650041e30 con 0x7fa65c129c70 2024-09-15T21:55:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f93137fe640 1 -- 172.21.15.47:0/3265366229 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c005c20 con 0x7f932c095a60 2024-09-15T21:55:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c11f5a0 con 0x7f932c095a60 2024-09-15T21:55:21.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c139b30 con 0x7f932c095a60 2024-09-15T21:55:21.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.252+0000 7fa64a7fc640 1 -- 172.21.15.47:0/730062048 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6500316f0 con 0x7fa65c129c70 2024-09-15T21:55:21.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.251+0000 7f1e38ff9640 1 -- 172.21.15.47:0/3300977634 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e300095a0 con 0x7f1e3c090460 2024-09-15T21:55:21.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.252+0000 7f93137fe640 1 -- 172.21.15.47:0/3265366229 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c0070f0 con 0x7f932c095a60 2024-09-15T21:55:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.252+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65c11d420 con 0x7fa65c129c70 2024-09-15T21:55:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.253+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c1392a0 con 0x7fa65c129c70 2024-09-15T21:55:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.252+0000 7f1e42f5d640 1 -- 172.21.15.47:0/3300977634 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3c06eb60 con 0x7f1e3c090460 2024-09-15T21:55:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.252+0000 7f1e42f5d640 1 -- 172.21.15.47:0/3300977634 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c12c210 con 0x7f1e3c090460 2024-09-15T21:55:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.252+0000 7f1e38ff9640 1 -- 172.21.15.47:0/3300977634 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3000c940 con 0x7f1e3c090460 2024-09-15T21:55:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.252+0000 7f1e38ff9640 1 -- 172.21.15.47:0/3300977634 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000cd00 con 0x7f1e3c090460 2024-09-15T21:55:21.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.253+0000 7f1e227fc640 1 -- 172.21.15.47:0/3300977634 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e3c04bf60 con 0x7f1e3c090460 2024-09-15T21:55:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.253+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf4004bf60 con 0x7fdf4012d660 2024-09-15T21:55:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.253+0000 7fdf277fe640 1 -- 172.21.15.47:0/3797624008 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fdf3803c630 con 0x7fdf4012d660 2024-09-15T21:55:21.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.253+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa65c04bf60 con 0x7fa65c129c70 2024-09-15T21:55:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.252+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f932c04bf60 con 0x7f932c095a60 2024-09-15T21:55:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.254+0000 7f93137fe640 1 -- 172.21.15.47:0/3265366229 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f931c007540 con 0x7f932c095a60 2024-09-15T21:55:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.254+0000 7fdf277fe640 1 -- 172.21.15.47:0/3797624008 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fdf38084090 con 0x7fdf4012d660 2024-09-15T21:55:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.254+0000 7fdf277fe640 1 -- 172.21.15.47:0/3797624008 --> v1:172.21.15.47:6809/32770 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf2c056ed0 con 0x7fdf2c0531f0 2024-09-15T21:55:21.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.254+0000 7fdf277fe640 1 -- 172.21.15.47:0/3797624008 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf380865f0 con 0x7fdf4012d660 2024-09-15T21:55:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.254+0000 7f1e38ff9640 1 -- 172.21.15.47:0/3300977634 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f1e30013680 con 0x7f1e3c090460 2024-09-15T21:55:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.254+0000 7fa64a7fc640 1 -- 172.21.15.47:0/730062048 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fa650031aa0 con 0x7fa65c129c70 2024-09-15T21:55:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.254+0000 7f1e38ff9640 1 -- 172.21.15.47:0/3300977634 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f1e30063ad0 con 0x7f1e3c090460 2024-09-15T21:55:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.255+0000 7f1e38ff9640 1 -- 172.21.15.47:0/3300977634 --> v1:172.21.15.47:6801/32760 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e28056ed0 con 0x7f1e280531f0 2024-09-15T21:55:21.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.255+0000 7f1e38ff9640 1 -- 172.21.15.47:0/3300977634 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e30068220 con 0x7f1e3c090460 2024-09-15T21:55:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.255+0000 7fa64a7fc640 1 -- 172.21.15.47:0/730062048 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fa65008b990 con 0x7fa65c129c70 2024-09-15T21:55:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.254+0000 7f93137fe640 1 -- 172.21.15.47:0/3265366229 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f931c062560 con 0x7f932c095a60 2024-09-15T21:55:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.254+0000 7f93137fe640 1 -- 172.21.15.47:0/3265366229 --> v1:172.21.15.47:6805/32761 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9318056fa0 con 0x7f93180532c0 2024-09-15T21:55:21.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.254+0000 7f93137fe640 1 -- 172.21.15.47:0/3265366229 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f931c064bf0 con 0x7f932c095a60 2024-09-15T21:55:21.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.255+0000 7fa64a7fc640 1 -- 172.21.15.47:0/730062048 --> v1:172.21.15.47:6813/32772 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa63c056d60 con 0x7fa63c053080 2024-09-15T21:55:21.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.255+0000 7fa64a7fc640 1 -- 172.21.15.47:0/730062048 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa650066940 con 0x7fa65c129c70 2024-09-15T21:55:21.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.255+0000 7fdf277fe640 1 -- 172.21.15.47:0/3797624008 <== osd.3 v1:172.21.15.47:6809/32770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fdf2c056ed0 con 0x7fdf2c0531f0 2024-09-15T21:55:21.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.256+0000 7fa64a7fc640 1 -- 172.21.15.47:0/730062048 <== osd.2 v1:172.21.15.47:6813/32772 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fa63c056d60 con 0x7fa63c053080 2024-09-15T21:55:21.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.256+0000 7f1e38ff9640 1 -- 172.21.15.47:0/3300977634 <== osd.0 v1:172.21.15.47:6801/32760 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f1e28056ed0 con 0x7f1e280531f0 2024-09-15T21:55:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.256+0000 7f93137fe640 1 -- 172.21.15.47:0/3265366229 <== osd.1 v1:172.21.15.47:6805/32761 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f9318056fa0 con 0x7f93180532c0 2024-09-15T21:55:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.268+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 --> v1:172.21.15.47:6809/32770 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf4008f9f0 con 0x7fdf2c0531f0 2024-09-15T21:55:21.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.268+0000 7fdf277fe640 1 -- 172.21.15.47:0/3797624008 <== osd.3 v1:172.21.15.47:6809/32770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7fdf4008f9f0 con 0x7fdf2c0531f0 2024-09-15T21:55:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 >> v1:172.21.15.47:6809/32770 conn(0x7fdf2c0531f0 legacy=0x7fdf2c055660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 >> v1:172.21.15.47:6800/32267 conn(0x7fdf2c04cb70 legacy=0x7fdf2c04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf4012d660 msgr2=0x7fdf4008f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7fdf45d00640 1 --2- 172.21.15.47:0/3797624008 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf4012d660 0x7fdf4008f270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009f20 tx=0x7fdf38036000 comp rx=0 tx=0).stop 2024-09-15T21:55:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7fdf3ffff640 1 -- 172.21.15.47:0/3797624008 reap_dead start 2024-09-15T21:55:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 shutdown_connections 2024-09-15T21:55:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 >> 172.21.15.47:0/3797624008 conn(0x7fdf40095a70 msgr2=0x7fdf4006e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 --> v1:172.21.15.47:6805/32761 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f932c13a0a0 con 0x7f93180532c0 2024-09-15T21:55:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 shutdown_connections 2024-09-15T21:55:21.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7fdf45d00640 1 -- 172.21.15.47:0/3797624008 wait complete. 2024-09-15T21:55:21.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.269+0000 7f93137fe640 1 -- 172.21.15.47:0/3265366229 <== osd.1 v1:172.21.15.47:6805/32761 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f932c13a0a0 con 0x7f93180532c0 2024-09-15T21:55:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.270+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 >> v1:172.21.15.47:6805/32761 conn(0x7f93180532c0 legacy=0x7f9318055730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.270+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 >> v1:172.21.15.47:6800/32267 conn(0x7f931804cc40 legacy=0x7f931804f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.270+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c095a60 msgr2=0x7f932c08b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.270+0000 7f933219d640 1 --2- 172.21.15.47:0/3265366229 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f932c095a60 0x7f932c08b880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c590 tx=0x7f931c007ee0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.270+0000 7f932bfff640 1 -- 172.21.15.47:0/3265366229 reap_dead start 2024-09-15T21:55:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.270+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 shutdown_connections 2024-09-15T21:55:21.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.270+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 >> 172.21.15.47:0/3265366229 conn(0x7f932c09ac50 msgr2=0x7f932c08f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.270+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 shutdown_connections 2024-09-15T21:55:21.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.270+0000 7f933219d640 1 -- 172.21.15.47:0/3265366229 wait complete. 2024-09-15T21:55:21.304 INFO:teuthology.orchestra.run.smithi047.stdout:38654705669 2024-09-15T21:55:21.305 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:55:21.305 INFO:teuthology.orchestra.run.smithi047.stdout:38654705668 2024-09-15T21:55:21.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:55:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.285+0000 7f1e227fc640 1 -- 172.21.15.47:0/3300977634 --> v1:172.21.15.47:6801/32760 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e3c08dbf0 con 0x7f1e280531f0 2024-09-15T21:55:21.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.285+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 --> v1:172.21.15.47:6813/32772 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa65c09bd80 con 0x7fa63c053080 2024-09-15T21:55:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.285+0000 7f1e38ff9640 1 -- 172.21.15.47:0/3300977634 <== osd.0 v1:172.21.15.47:6801/32760 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f1e3c08dbf0 con 0x7f1e280531f0 2024-09-15T21:55:21.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.285+0000 7fa64a7fc640 1 -- 172.21.15.47:0/730062048 <== osd.2 v1:172.21.15.47:6813/32772 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7fa65c09bd80 con 0x7fa63c053080 2024-09-15T21:55:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7f1e227fc640 1 -- 172.21.15.47:0/3300977634 >> v1:172.21.15.47:6801/32760 conn(0x7f1e280531f0 legacy=0x7f1e28055660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7f1e227fc640 1 -- 172.21.15.47:0/3300977634 >> v1:172.21.15.47:6800/32267 conn(0x7f1e2804cb70 legacy=0x7f1e2804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7f1e227fc640 1 -- 172.21.15.47:0/3300977634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e3c090460 msgr2=0x7f1e3c08c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7f1e227fc640 1 --2- 172.21.15.47:0/3300977634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e3c090460 0x7f1e3c08c940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000ea50 tx=0x7f1e300091d0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7f1e414fa640 1 -- 172.21.15.47:0/3300977634 reap_dead start 2024-09-15T21:55:21.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 >> v1:172.21.15.47:6813/32772 conn(0x7fa63c053080 legacy=0x7fa63c0554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7f1e227fc640 1 -- 172.21.15.47:0/3300977634 shutdown_connections 2024-09-15T21:55:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7f1e227fc640 1 -- 172.21.15.47:0/3300977634 >> 172.21.15.47:0/3300977634 conn(0x7f1e3c095a70 msgr2=0x7f1e3c06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 >> v1:172.21.15.47:6800/32267 conn(0x7fa63c04cb70 legacy=0x7fa63c04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa65c129c70 msgr2=0x7fa65c11a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7fa660d5d640 1 --2- 172.21.15.47:0/730062048 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa65c129c70 0x7fa65c11a180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa650031000 tx=0x7fa650032000 comp rx=0 tx=0).stop 2024-09-15T21:55:21.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7f1e227fc640 1 -- 172.21.15.47:0/3300977634 shutdown_connections 2024-09-15T21:55:21.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7fa65ad9d640 1 -- 172.21.15.47:0/730062048 reap_dead start 2024-09-15T21:55:21.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7f1e227fc640 1 -- 172.21.15.47:0/3300977634 wait complete. 2024-09-15T21:55:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 shutdown_connections 2024-09-15T21:55:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 >> 172.21.15.47:0/730062048 conn(0x7fa65c09d850 msgr2=0x7fa65c09b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.286+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 shutdown_connections 2024-09-15T21:55:21.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.287+0000 7fa660d5d640 1 -- 172.21.15.47:0/730062048 wait complete. 2024-09-15T21:55:21.311 INFO:teuthology.orchestra.run.smithi047.stdout:38654705668 2024-09-15T21:55:21.312 INFO:teuthology.orchestra.run.smithi047.stdout:38654705669 2024-09-15T21:55:21.312 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:55:21.313 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:55:21.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.445+0000 7f5857a6f640 1 Processor -- start 2024-09-15T21:55:21.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.445+0000 7f55cb59d640 1 Processor -- start 2024-09-15T21:55:21.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.445+0000 7f7c8519d640 1 Processor -- start 2024-09-15T21:55:21.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f7c8519d640 1 -- start start 2024-09-15T21:55:21.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f55cb59d640 1 -- start start 2024-09-15T21:55:21.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f253a71d640 1 Processor -- start 2024-09-15T21:55:21.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f5857a6f640 1 -- start start 2024-09-15T21:55:21.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f253a71d640 1 -- start start 2024-09-15T21:55:21.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f7c8519d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c80129c70 0x7f7c8012a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f5857a6f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850095a60 0x7f5850095e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f55cb59d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 0x7f55c4127d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f7c8519d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c800970f0 0x7f7c80097570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f5857a6f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f585012f920 0x7f5850131d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f55cb59d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f55c4128240 0x7f55c4087f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f55cb59d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c4127ad0 con 0x7f55c4128670 2024-09-15T21:55:21.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f55cb59d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c408df30 con 0x7f55c4128240 2024-09-15T21:55:21.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f55cb59d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4089e40 con 0x7f55c406eec0 2024-09-15T21:55:21.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f7c8519d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c800998b0 con 0x7f7c8012a580 2024-09-15T21:55:21.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f7c8519d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8009a180 con 0x7f7c80129c70 2024-09-15T21:55:21.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f7c8519d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8009a440 con 0x7f7c800970f0 2024-09-15T21:55:21.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f7c7ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c80129c70 0x7f7c8012a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f5857a6f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58500689e0 con 0x7f5850096370 2024-09-15T21:55:21.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f5857a6f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5850068140 con 0x7f585012f920 2024-09-15T21:55:21.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f7c7ed9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7c80129c70 0x7f7c8012a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47830/0 (socket says 172.21.15.47:47830) 2024-09-15T21:55:21.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f7c7e59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7c8012a580 0x7f7c800969e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:52996/0 (socket says 172.21.15.47:52996) 2024-09-15T21:55:21.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f7c7ed9d640 1 -- 172.21.15.47:0/783039003 learned_addr learned my addr 172.21.15.47:0/783039003 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f585500a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5850096370 0x7f585012e200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53012/0 (socket says 172.21.15.47:53012) 2024-09-15T21:55:21.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f585500a640 1 -- 172.21.15.47:0/2294763721 learned_addr learned my addr 172.21.15.47:0/2294763721 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:21.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f5857a6f640 1 -- 172.21.15.47:0/2294763721 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5850069010 con 0x7f5850095a60 2024-09-15T21:55:21.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f253a71d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f2534096520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f253a71d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2534096a60 0x7f2534096e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f585580b640 1 --2- 172.21.15.47:0/2294763721 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850095a60 0x7f5850095e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f253a71d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25340907c0 con 0x7f253412c250 2024-09-15T21:55:21.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f253a71d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25340900a0 con 0x7f2534096a60 2024-09-15T21:55:21.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f7c7f59e640 1 --2- 172.21.15.47:0/783039003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c800970f0 0x7f7c80097570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.446+0000 7f253a71d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f253408fa90 con 0x7f2534096150 2024-09-15T21:55:21.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f2533fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f2534096520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f2533fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f2534096520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48074/0 (socket says 172.21.15.47:48074) 2024-09-15T21:55:21.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f2533fff640 1 -- 172.21.15.47:0/2584418621 learned_addr learned my addr 172.21.15.47:0/2584418621 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f25337fe640 1 --2- 172.21.15.47:0/2584418621 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2534096a60 0x7f2534096e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55c9b3a640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f55c4128670 0x7f55c40884b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53020/0 (socket says 172.21.15.47:53020) 2024-09-15T21:55:21.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55c9b3a640 1 -- 172.21.15.47:0/1508122730 learned_addr learned my addr 172.21.15.47:0/1508122730 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:21.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55c8b38640 1 --2- 172.21.15.47:0/1508122730 >> v2:172.21.15.47:3300/0 conn(0x7f55c4128240 0x7f55c4087f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55c9339640 1 --2- 172.21.15.47:0/1508122730 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 0x7f55c4127d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55bbfff640 1 -- 172.21.15.47:0/1508122730 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439417913 0 0) 0x7f55c4127ad0 con 0x7f55c4128670 2024-09-15T21:55:21.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55bbfff640 1 -- 172.21.15.47:0/1508122730 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b00034e0 con 0x7f55c4128670 2024-09-15T21:55:21.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f7c7ed9d640 1 -- 172.21.15.47:0/783039003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c800970f0 msgr2=0x7f7c80097570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f7c7ed9d640 1 --2- 172.21.15.47:0/783039003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c800970f0 0x7f7c80097570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f7c7ed9d640 1 -- 172.21.15.47:0/783039003 >> v1:172.21.15.47:6789/0 conn(0x7f7c8012a580 legacy=0x7f7c800969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f7c7ed9d640 1 -- 172.21.15.47:0/783039003 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80099db0 con 0x7f7c80129c70 2024-09-15T21:55:21.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55c8b38640 1 -- 172.21.15.47:0/1508122730 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 msgr2=0x7f55c4127d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55c8b38640 1 --2- 172.21.15.47:0/1508122730 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 0x7f55c4127d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55c8b38640 1 -- 172.21.15.47:0/1508122730 >> v1:172.21.15.47:6789/0 conn(0x7f55c4128670 legacy=0x7f55c40884b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55c8b38640 1 -- 172.21.15.47:0/1508122730 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c408a5b0 con 0x7f55c4128240 2024-09-15T21:55:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f55c9339640 1 --2- 172.21.15.47:0/1508122730 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 0x7f55c4127d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f585580b640 1 -- 172.21.15.47:0/2294763721 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f585012f920 msgr2=0x7f5850131d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f585580b640 1 --2- 172.21.15.47:0/2294763721 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f585012f920 0x7f5850131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f585580b640 1 -- 172.21.15.47:0/2294763721 >> v1:172.21.15.47:6789/0 conn(0x7f5850096370 legacy=0x7f585012e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.448+0000 7f25337fe640 1 -- 172.21.15.47:0/2584418621 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 msgr2=0x7f2534096520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:21.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.447+0000 7f585580b640 1 -- 172.21.15.47:0/2294763721 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585006ec40 con 0x7f5850095a60 2024-09-15T21:55:21.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.448+0000 7f25337fe640 1 --2- 172.21.15.47:0/2584418621 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f2534096520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.448+0000 7f25337fe640 1 -- 172.21.15.47:0/2584418621 >> v1:172.21.15.47:6789/0 conn(0x7f253412c250 legacy=0x7f253412e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.448+0000 7f2532ffd640 1 -- 172.21.15.47:0/2584418621 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114045358 0 0) 0x7f25340907c0 con 0x7f253412c250 2024-09-15T21:55:21.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.448+0000 7f25337fe640 1 -- 172.21.15.47:0/2584418621 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2534091020 con 0x7f2534096a60 2024-09-15T21:55:21.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.448+0000 7f2533fff640 1 --2- 172.21.15.47:0/2584418621 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f2534096520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:21.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.448+0000 7f55c8b38640 1 --2- 172.21.15.47:0/1508122730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 0x7f55c4087f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009850 tx=0x7f55b40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cf904b44da269a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.448+0000 7f585580b640 1 --2- 172.21.15.47:0/2294763721 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850095a60 0x7f5850095e30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f584c0098b0 tx=0x7f584c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0902ef151b6ef34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.448+0000 7f7c7ed9d640 1 --2- 172.21.15.47:0/783039003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c80129c70 0x7f7c8012a040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009910 tx=0x7f7c700091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d353ad3767d6e1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.448+0000 7f55bbfff640 1 -- 172.21.15.47:0/1508122730 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b402ac60 con 0x7f55c4128240 2024-09-15T21:55:21.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f5854809640 1 -- 172.21.15.47:0/2294763721 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c02ac30 con 0x7f5850095a60 2024-09-15T21:55:21.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f55bbfff640 1 -- 172.21.15.47:0/1508122730 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b402adc0 con 0x7f55c4128240 2024-09-15T21:55:21.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f55bbfff640 1 -- 172.21.15.47:0/1508122730 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4032670 con 0x7f55c4128240 2024-09-15T21:55:21.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f5854809640 1 -- 172.21.15.47:0/2294763721 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c02ad90 con 0x7f5850095a60 2024-09-15T21:55:21.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f5854809640 1 -- 172.21.15.47:0/2294763721 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c032670 con 0x7f5850095a60 2024-09-15T21:55:21.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f7c7dd9b640 1 -- 172.21.15.47:0/783039003 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7002ac90 con 0x7f7c80129c70 2024-09-15T21:55:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f7c7dd9b640 1 -- 172.21.15.47:0/783039003 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7002adf0 con 0x7f7c80129c70 2024-09-15T21:55:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f7c7dd9b640 1 -- 172.21.15.47:0/783039003 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70032670 con 0x7f7c80129c70 2024-09-15T21:55:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f25337fe640 1 --2- 172.21.15.47:0/2584418621 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2534096a60 0x7f2534096e50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2528009880 tx=0x7f25280090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7477106d17d44637 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f2532ffd640 1 -- 172.21.15.47:0/2584418621 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252802ad60 con 0x7f2534096a60 2024-09-15T21:55:21.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f2532ffd640 1 -- 172.21.15.47:0/2584418621 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2528032440 con 0x7f2534096a60 2024-09-15T21:55:21.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.449+0000 7f2532ffd640 1 -- 172.21.15.47:0/2584418621 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25280327e0 con 0x7f2534096a60 2024-09-15T21:55:21.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f5857a6f640 1 -- 172.21.15.47:0/2294763721 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850095a60 msgr2=0x7f5850095e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f5857a6f640 1 --2- 172.21.15.47:0/2294763721 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850095a60 0x7f5850095e30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f584c0098b0 tx=0x7f584c0090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f55cb59d640 1 -- 172.21.15.47:0/1508122730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 msgr2=0x7f55c4087f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f55cb59d640 1 --2- 172.21.15.47:0/1508122730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 0x7f55c4087f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009850 tx=0x7f55b40090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f7c8519d640 1 -- 172.21.15.47:0/783039003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c80129c70 msgr2=0x7f7c8012a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f5857a6f640 1 -- 172.21.15.47:0/2294763721 shutdown_connections 2024-09-15T21:55:21.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f5857a6f640 1 --2- 172.21.15.47:0/2294763721 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f585012f920 0x7f5850131d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f5857a6f640 1 --2- 172.21.15.47:0/2294763721 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850095a60 0x7f5850095e30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f5857a6f640 1 -- 172.21.15.47:0/2294763721 >> 172.21.15.47:0/2294763721 conn(0x7f585009ac50 msgr2=0x7f5850096b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f7c8519d640 1 --2- 172.21.15.47:0/783039003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c80129c70 0x7f7c8012a040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009910 tx=0x7f7c700091d0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f55cb59d640 1 -- 172.21.15.47:0/1508122730 shutdown_connections 2024-09-15T21:55:21.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f55cb59d640 1 --2- 172.21.15.47:0/1508122730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 0x7f55c4087f70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f55cb59d640 1 --2- 172.21.15.47:0/1508122730 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 0x7f55c4127d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f55cb59d640 1 -- 172.21.15.47:0/1508122730 >> 172.21.15.47:0/1508122730 conn(0x7f55c409b700 msgr2=0x7f55c409bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f55cb59d640 1 -- 172.21.15.47:0/1508122730 shutdown_connections 2024-09-15T21:55:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f7c8519d640 1 -- 172.21.15.47:0/783039003 shutdown_connections 2024-09-15T21:55:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f7c8519d640 1 --2- 172.21.15.47:0/783039003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c800970f0 0x7f7c80097570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f7c8519d640 1 --2- 172.21.15.47:0/783039003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c80129c70 0x7f7c8012a040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f7c8519d640 1 -- 172.21.15.47:0/783039003 >> 172.21.15.47:0/783039003 conn(0x7f7c8009d850 msgr2=0x7f7c8009dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f5857a6f640 1 -- 172.21.15.47:0/2294763721 shutdown_connections 2024-09-15T21:55:21.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f55cb59d640 1 -- 172.21.15.47:0/1508122730 wait complete. 2024-09-15T21:55:21.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f253a71d640 1 -- 172.21.15.47:0/2584418621 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2534096a60 msgr2=0x7f2534096e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f253a71d640 1 --2- 172.21.15.47:0/2584418621 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2534096a60 0x7f2534096e50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2528009880 tx=0x7f25280090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f5857a6f640 1 -- 172.21.15.47:0/2294763721 wait complete. 2024-09-15T21:55:21.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f253a71d640 1 -- 172.21.15.47:0/2584418621 shutdown_connections 2024-09-15T21:55:21.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f7c8519d640 1 -- 172.21.15.47:0/783039003 shutdown_connections 2024-09-15T21:55:21.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f253a71d640 1 --2- 172.21.15.47:0/2584418621 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f2534096a60 0x7f2534096e50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f7c8519d640 1 -- 172.21.15.47:0/783039003 wait complete. 2024-09-15T21:55:21.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f253a71d640 1 --2- 172.21.15.47:0/2584418621 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f2534096520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f253a71d640 1 -- 172.21.15.47:0/2584418621 >> 172.21.15.47:0/2584418621 conn(0x7f253409b450 msgr2=0x7f253409b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f253a71d640 1 -- 172.21.15.47:0/2584418621 shutdown_connections 2024-09-15T21:55:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.450+0000 7f253a71d640 1 -- 172.21.15.47:0/2584418621 wait complete. 2024-09-15T21:55:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f5857a6f640 1 Processor -- start 2024-09-15T21:55:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f5857a6f640 1 -- start start 2024-09-15T21:55:21.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f55cb59d640 1 Processor -- start 2024-09-15T21:55:21.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f55cb59d640 1 -- start start 2024-09-15T21:55:21.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f7c8519d640 1 Processor -- start 2024-09-15T21:55:21.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f7c8519d640 1 -- start start 2024-09-15T21:55:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f5857a6f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850095a60 0x7f585012f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f5857a6f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850096370 0x7f585008bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f5857a6f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f585012f920 0x7f585008c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f5857a6f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58500983c0 con 0x7f585012f920 2024-09-15T21:55:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f5857a6f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5850120090 con 0x7f5850095a60 2024-09-15T21:55:21.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f253a71d640 1 Processor -- start 2024-09-15T21:55:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.451+0000 7f5857a6f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5850120460 con 0x7f5850096370 2024-09-15T21:55:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585580b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850095a60 0x7f585012f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55cb59d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 0x7f55c4122480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55cb59d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 0x7f55c41229c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c8519d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c800970f0 0x7f7c801393e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55cb59d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f55c4128670 0x7f55c4122f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585600c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f585012f920 0x7f585008c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55cb59d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4124660 con 0x7f55c4128240 2024-09-15T21:55:21.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55cb59d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4125160 con 0x7f55c4128670 2024-09-15T21:55:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585580b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850095a60 0x7f585012f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52378/0 (socket says 172.21.15.47:52378) 2024-09-15T21:55:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585580b640 1 -- 172.21.15.47:0/1075593751 learned_addr learned my addr 172.21.15.47:0/1075593751 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585500a640 1 --2- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850096370 0x7f585008bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55c8b38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 0x7f55c41229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c8519d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c80129c70 0x7f7c80139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c8519d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7c8012a580 0x7f7c8011a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c8519d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8011d210 con 0x7f7c800970f0 2024-09-15T21:55:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c8519d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8011c590 con 0x7f7c80129c70 2024-09-15T21:55:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c8519d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8011ddc0 con 0x7f7c8012a580 2024-09-15T21:55:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c800970f0 0x7f7c801393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c800970f0 0x7f7c801393e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47854/0 (socket says 172.21.15.47:47854) 2024-09-15T21:55:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7ed9d640 1 -- 172.21.15.47:0/3055398344 learned_addr learned my addr 172.21.15.47:0/3055398344 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55c8b38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 0x7f55c41229c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47850/0 (socket says 172.21.15.47:47850) 2024-09-15T21:55:21.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55c8b38640 1 -- 172.21.15.47:0/1718185477 learned_addr learned my addr 172.21.15.47:0/1718185477 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55c9b3a640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f55c4128670 0x7f55c4122f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55c4124be0 con 0x7f55c406eec0 2024-09-15T21:55:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585600c640 1 -- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850096370 msgr2=0x7f585008bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585600c640 1 --2- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850096370 0x7f585008bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585600c640 1 -- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850095a60 msgr2=0x7f585012f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585600c640 1 --2- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850095a60 0x7f585012f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585600c640 1 -- 172.21.15.47:0/1075593751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c009390 con 0x7f585012f920 2024-09-15T21:55:21.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f55c9339640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 0x7f55c4122480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7f59e640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7c8012a580 0x7f7c8011a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7e59c640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c80129c70 0x7f7c80139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7ed9d640 1 -- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7c8012a580 msgr2=0x7f7c8011a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7ed9d640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7c8012a580 0x7f7c8011a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7ed9d640 1 -- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c80129c70 msgr2=0x7f7c80139920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7ed9d640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c80129c70 0x7f7c80139920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7ed9d640 1 -- 172.21.15.47:0/3055398344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70009450 con 0x7f7c800970f0 2024-09-15T21:55:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7e59c640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c80129c70 0x7f7c80139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:55:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f7c7f59e640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7c8012a580 0x7f7c8011a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:55:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f585600c640 1 --2- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f585012f920 0x7f585008c4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5848009fd0 tx=0x7f5848009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f7c7ed9d640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c800970f0 0x7f7c801393e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009420 tx=0x7f7c70032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55c9b3a640 1 -- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 msgr2=0x7f55c4122480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55c9b3a640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 0x7f55c4122480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55c9b3a640 1 -- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 msgr2=0x7f55c41229c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55c9b3a640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 0x7f55c41229c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55c9b3a640 1 -- 172.21.15.47:0/1718185477 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c0009390 con 0x7f55c4128670 2024-09-15T21:55:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55c8b38640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 0x7f55c41229c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.452+0000 7f253a71d640 1 -- start start 2024-09-15T21:55:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f253a71d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f253412b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f253a71d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2534096a60 0x7f253412b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f253a71d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f253412c250 0x7f253412bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:21.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55c9339640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 0x7f55c4122480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f2533fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f253412b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3055398344 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70006b30 con 0x7f7c800970f0 2024-09-15T21:55:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f5845ffb640 1 -- 172.21.15.47:0/1075593751 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584800c950 con 0x7f585012f920 2024-09-15T21:55:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f5845ffb640 1 -- 172.21.15.47:0/1075593751 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584800cad0 con 0x7f585012f920 2024-09-15T21:55:21.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3055398344 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c70032e80 con 0x7f7c800970f0 2024-09-15T21:55:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f5845ffb640 1 -- 172.21.15.47:0/1075593751 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848014410 con 0x7f585012f920 2024-09-15T21:55:21.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3055398344 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700317e0 con 0x7f7c800970f0 2024-09-15T21:55:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f253a71d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253408ce10 con 0x7f2534096a60 2024-09-15T21:55:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f253a71d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253408b9e0 con 0x7f253412c250 2024-09-15T21:55:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f253a71d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f253408b110 con 0x7f2534096150 2024-09-15T21:55:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f2533fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f253412b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48120/0 (socket says 172.21.15.47:48120) 2024-09-15T21:55:21.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f2533fff640 1 -- 172.21.15.47:0/3164249801 learned_addr learned my addr 172.21.15.47:0/3164249801 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55c9b3a640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f55c4128670 0x7f55c4122f00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00cbc0 tx=0x7f55bc001a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55b97fa640 1 -- 172.21.15.47:0/1718185477 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00a640 con 0x7f55c4128670 2024-09-15T21:55:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55b97fa640 1 -- 172.21.15.47:0/1718185477 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc00a7a0 con 0x7f55c4128670 2024-09-15T21:55:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f55b97fa640 1 -- 172.21.15.47:0/1718185477 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00aba0 con 0x7f55c4128670 2024-09-15T21:55:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b4009390 con 0x7f55c4128670 2024-09-15T21:55:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c408da10 con 0x7f55c4128670 2024-09-15T21:55:21.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f7c8519d640 1 -- 172.21.15.47:0/3055398344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8011d910 con 0x7f7c800970f0 2024-09-15T21:55:21.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f7c8519d640 1 -- 172.21.15.47:0/3055398344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c80139e60 con 0x7f7c800970f0 2024-09-15T21:55:21.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f5857a6f640 1 -- 172.21.15.47:0/1075593751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585008d790 con 0x7f585012f920 2024-09-15T21:55:21.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f5857a6f640 1 -- 172.21.15.47:0/1075593751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585008cc40 con 0x7f585012f920 2024-09-15T21:55:21.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f2538cba640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f253412c250 0x7f253412bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.453+0000 7f25337fe640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2534096a60 0x7f253412b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:21.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f25337fe640 1 -- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 msgr2=0x7f253412b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f25337fe640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f253412b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f25337fe640 1 -- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f253412c250 msgr2=0x7f253412bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f25337fe640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f253412c250 0x7f253412bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f25337fe640 1 -- 172.21.15.47:0/3164249801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524009450 con 0x7f2534096a60 2024-09-15T21:55:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f2538cba640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f253412c250 0x7f253412bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:55:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.454+0000 7f2533fff640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f253412b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.455+0000 7f25337fe640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2534096a60 0x7f253412b7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2534090230 tx=0x7f2528032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.455+0000 7f251bfff640 1 -- 172.21.15.47:0/3164249801 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528031de0 con 0x7f2534096a60 2024-09-15T21:55:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.455+0000 7f251bfff640 1 -- 172.21.15.47:0/3164249801 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252803d440 con 0x7f2534096a60 2024-09-15T21:55:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.455+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c4124340 con 0x7f55c4128670 2024-09-15T21:55:21.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.456+0000 7f55b97fa640 1 -- 172.21.15.47:0/1718185477 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f55bc0070f0 con 0x7f55c4128670 2024-09-15T21:55:21.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.456+0000 7f5845ffb640 1 -- 172.21.15.47:0/1075593751 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f5848014840 con 0x7f585012f920 2024-09-15T21:55:21.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.456+0000 7f7c8519d640 1 -- 172.21.15.47:0/3055398344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8004bf60 con 0x7f7c800970f0 2024-09-15T21:55:21.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.456+0000 7f251bfff640 1 -- 172.21.15.47:0/3164249801 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252803d7e0 con 0x7f2534096a60 2024-09-15T21:55:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.456+0000 7f55b97fa640 1 -- 172.21.15.47:0/1718185477 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f55bc063320 con 0x7f55c4128670 2024-09-15T21:55:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.456+0000 7f253a71d640 1 -- 172.21.15.47:0/3164249801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2528009390 con 0x7f2534096a60 2024-09-15T21:55:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.456+0000 7f5857a6f640 1 -- 172.21.15.47:0/1075593751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585011f810 con 0x7f585012f920 2024-09-15T21:55:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.456+0000 7f5845ffb640 1 -- 172.21.15.47:0/1075593751 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f5848063100 con 0x7f585012f920 2024-09-15T21:55:21.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.456+0000 7f253a71d640 1 -- 172.21.15.47:0/3164249801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253408ea50 con 0x7f2534096a60 2024-09-15T21:55:21.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.456+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3055398344 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f7c70031b90 con 0x7f7c800970f0 2024-09-15T21:55:21.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.457+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3055398344 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f7c7008bb20 con 0x7f7c800970f0 2024-09-15T21:55:21.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.457+0000 7f253a71d640 1 -- 172.21.15.47:0/3164249801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253408c1c0 con 0x7f2534096a60 2024-09-15T21:55:21.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.458+0000 7f251bfff640 1 -- 172.21.15.47:0/3164249801 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f2528039040 con 0x7f2534096a60 2024-09-15T21:55:21.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.458+0000 7f251bfff640 1 -- 172.21.15.47:0/3164249801 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f25280833e0 con 0x7f2534096a60 2024-09-15T21:55:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.460+0000 7f55b97fa640 1 -- 172.21.15.47:0/1718185477 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f55bc03e400 con 0x7f55c4128670 2024-09-15T21:55:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.463+0000 7f5845ffb640 1 -- 172.21.15.47:0/1075593751 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f585011f810 con 0x7f585012f920 2024-09-15T21:55:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.465+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3055398344 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f7c70066c60 con 0x7f7c800970f0 2024-09-15T21:55:21.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.468+0000 7f251bfff640 1 -- 172.21.15.47:0/3164249801 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f253408c1c0 con 0x7f2534096a60 2024-09-15T21:55:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.625+0000 7f5857a6f640 1 -- 172.21.15.47:0/1075593751 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f585011f810 con 0x7f585012f920 2024-09-15T21:55:21.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.626+0000 7f5845ffb640 1 -- 172.21.15.47:0/1075593751 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f585011f810 con 0x7f585012f920 2024-09-15T21:55:21.628 INFO:teuthology.orchestra.run.smithi047.stdout:38654705666 2024-09-15T21:55:21.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.626+0000 7f7c8519d640 1 -- 172.21.15.47:0/3055398344 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7c8004bf60 con 0x7f7c800970f0 2024-09-15T21:55:21.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.626+0000 7f7c66ffd640 1 -- 172.21.15.47:0/3055398344 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c70037030 con 0x7f7c800970f0 2024-09-15T21:55:21.629 INFO:teuthology.orchestra.run.smithi047.stdout:38654705667 2024-09-15T21:55:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.628+0000 7f5857a6f640 1 -- 172.21.15.47:0/1075593751 >> v1:172.21.15.47:6800/32267 conn(0x7f583404ccf0 legacy=0x7f583404f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.628+0000 7f5857a6f640 1 -- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f585012f920 msgr2=0x7f585008c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.628+0000 7f5857a6f640 1 --2- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f585012f920 0x7f585008c4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5848009fd0 tx=0x7f5848009780 comp rx=0 tx=0).stop 2024-09-15T21:55:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.628+0000 7f5857a6f640 1 -- 172.21.15.47:0/1075593751 shutdown_connections 2024-09-15T21:55:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.628+0000 7f5857a6f640 1 --2- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f585012f920 0x7f585008c4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.628+0000 7f5857a6f640 1 --2- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5850096370 0x7f585008bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.628+0000 7f5857a6f640 1 --2- 172.21.15.47:0/1075593751 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5850095a60 0x7f585012f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.628+0000 7f5857a6f640 1 -- 172.21.15.47:0/1075593751 >> 172.21.15.47:0/1075593751 conn(0x7f585009ac50 msgr2=0x7f5850091520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.628+0000 7f5857a6f640 1 -- 172.21.15.47:0/1075593751 shutdown_connections 2024-09-15T21:55:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.628+0000 7f5857a6f640 1 -- 172.21.15.47:0/1075593751 wait complete. 2024-09-15T21:55:21.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.629+0000 7f7c8519d640 1 -- 172.21.15.47:0/3055398344 >> v1:172.21.15.47:6800/32267 conn(0x7f7c6c04cba0 legacy=0x7f7c6c04f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.629+0000 7f7c8519d640 1 -- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c800970f0 msgr2=0x7f7c801393e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.629+0000 7f7c8519d640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c800970f0 0x7f7c801393e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009420 tx=0x7f7c70032000 comp rx=0 tx=0).stop 2024-09-15T21:55:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.629+0000 7f7c8519d640 1 -- 172.21.15.47:0/3055398344 shutdown_connections 2024-09-15T21:55:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.629+0000 7f7c8519d640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7c8012a580 0x7f7c8011a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.629+0000 7f7c8519d640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7c80129c70 0x7f7c80139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.629+0000 7f7c8519d640 1 --2- 172.21.15.47:0/3055398344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c800970f0 0x7f7c801393e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.629+0000 7f7c8519d640 1 -- 172.21.15.47:0/3055398344 >> 172.21.15.47:0/3055398344 conn(0x7f7c8009d850 msgr2=0x7f7c8012cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.629+0000 7f7c8519d640 1 -- 172.21.15.47:0/3055398344 shutdown_connections 2024-09-15T21:55:21.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.629+0000 7f7c8519d640 1 -- 172.21.15.47:0/3055398344 wait complete. 2024-09-15T21:55:21.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705666 for osd.0 2024-09-15T21:55:21.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705667 for osd.3 2024-09-15T21:55:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.659+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f55c404bf60 con 0x7f55c4128670 2024-09-15T21:55:21.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.659+0000 7f55b97fa640 1 -- 172.21.15.47:0/1718185477 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55bc041c10 con 0x7f55c4128670 2024-09-15T21:55:21.661 INFO:teuthology.orchestra.run.smithi047.stdout:38654705666 2024-09-15T21:55:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.662+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 >> v1:172.21.15.47:6800/32267 conn(0x7f55a804cbf0 legacy=0x7f55a804f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.662+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f55c4128670 msgr2=0x7f55c4122f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.662+0000 7f55cb59d640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f55c4128670 0x7f55c4122f00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00cbc0 tx=0x7f55bc001a20 comp rx=0 tx=0).stop 2024-09-15T21:55:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.662+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 shutdown_connections 2024-09-15T21:55:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.662+0000 7f55cb59d640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f55c4128670 0x7f55c4122f00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.662+0000 7f55cb59d640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55c4128240 0x7f55c41229c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.662+0000 7f55cb59d640 1 --2- 172.21.15.47:0/1718185477 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f55c406eec0 0x7f55c4122480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.662+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 >> 172.21.15.47:0/1718185477 conn(0x7f55c409b700 msgr2=0x7f55c40691a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.662+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 shutdown_connections 2024-09-15T21:55:21.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.662+0000 7f55cb59d640 1 -- 172.21.15.47:0/1718185477 wait complete. 2024-09-15T21:55:21.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.666+0000 7f253a71d640 1 -- 172.21.15.47:0/3164249801 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f253408c1c0 con 0x7f2534096a60 2024-09-15T21:55:21.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.666+0000 7f251bfff640 1 -- 172.21.15.47:0/3164249801 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f253408c1c0 con 0x7f2534096a60 2024-09-15T21:55:21.668 INFO:teuthology.orchestra.run.smithi047.stdout:38654705667 2024-09-15T21:55:21.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.668+0000 7f253a71d640 1 -- 172.21.15.47:0/3164249801 >> v1:172.21.15.47:6800/32267 conn(0x7f252004cc40 legacy=0x7f252004f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.668+0000 7f253a71d640 1 -- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2534096a60 msgr2=0x7f253412b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.668+0000 7f253a71d640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2534096a60 0x7f253412b7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2534090230 tx=0x7f2528032000 comp rx=0 tx=0).stop 2024-09-15T21:55:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.668+0000 7f253a71d640 1 -- 172.21.15.47:0/3164249801 shutdown_connections 2024-09-15T21:55:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.668+0000 7f253a71d640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f253412c250 0x7f253412bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.668+0000 7f253a71d640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2534096a60 0x7f253412b7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.668+0000 7f253a71d640 1 --2- 172.21.15.47:0/3164249801 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f2534096150 0x7f253412b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.668+0000 7f253a71d640 1 -- 172.21.15.47:0/3164249801 >> 172.21.15.47:0/3164249801 conn(0x7f253409b450 msgr2=0x7f2534094e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.668+0000 7f253a71d640 1 -- 172.21.15.47:0/3164249801 shutdown_connections 2024-09-15T21:55:21.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:21.668+0000 7f253a71d640 1 -- 172.21.15.47:0/3164249801 wait complete. 2024-09-15T21:55:21.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705666 for osd.1 2024-09-15T21:55:21.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705667 for osd.2 2024-09-15T21:55:22.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:55:22.649 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:55:22.676 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:55:22.682 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:55:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.761+0000 7f1597b00640 1 Processor -- start 2024-09-15T21:55:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.761+0000 7f0a7e31d640 1 Processor -- start 2024-09-15T21:55:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.761+0000 7f0a7e31d640 1 -- start start 2024-09-15T21:55:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.761+0000 7f1597b00640 1 -- start start 2024-09-15T21:55:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a7e31d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a780957f0 0x7f0a78095bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f1597b00640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590129c70 0x7f159012a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a7e31d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a7812c230 0x7f0a7812e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f1597b00640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 0x7f15900969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a7e31d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7808fd20 con 0x7f0a78090d70 2024-09-15T21:55:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a7e31d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a78096a40 con 0x7f0a7812c230 2024-09-15T21:55:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a7e31d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a78090230 con 0x7f0a780957f0 2024-09-15T21:55:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f1597b00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1590096f20 0x7f1590097370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f1597b00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15900998b0 con 0x7f1590096f20 2024-09-15T21:55:22.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f1597b00640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159009a180 con 0x7f1590129c70 2024-09-15T21:55:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a77fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0a78090d70 0x7f0a780950e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53036/0 (socket says 172.21.15.47:53036) 2024-09-15T21:55:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a77fff640 1 -- 172.21.15.47:0/2036932342 learned_addr learned my addr 172.21.15.47:0/2036932342 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a777fe640 1 --2- 172.21.15.47:0/2036932342 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a780957f0 0x7f0a78095bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f1597b00640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f159009a440 con 0x7f159012a580 2024-09-15T21:55:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f159589c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590129c70 0x7f159012a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f159509b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 0x7f15900969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f159589c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590129c70 0x7f159012a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52432/0 (socket says 172.21.15.47:52432) 2024-09-15T21:55:22.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f159589c640 1 -- 172.21.15.47:0/2136759661 learned_addr learned my addr 172.21.15.47:0/2136759661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:22.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f159509b640 1 --2- 172.21.15.47:0/2136759661 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 0x7f15900969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48132/0 (socket says 172.21.15.47:48132) 2024-09-15T21:55:22.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a7c8ba640 1 --2- 172.21.15.47:0/2036932342 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a7812c230 0x7f0a7812e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f159609d640 1 --2- 172.21.15.47:0/2136759661 >> v2:172.21.15.47:3300/0 conn(0x7f1590096f20 0x7f1590097370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f159589c640 1 -- 172.21.15.47:0/2136759661 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 msgr2=0x7f15900969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f159589c640 1 --2- 172.21.15.47:0/2136759661 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 0x7f15900969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a777fe640 1 -- 172.21.15.47:0/2036932342 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a7812c230 msgr2=0x7f0a7812e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.762+0000 7f0a777fe640 1 --2- 172.21.15.47:0/2036932342 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a7812c230 0x7f0a7812e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.763+0000 7f159589c640 1 -- 172.21.15.47:0/2136759661 >> v2:172.21.15.47:3300/0 conn(0x7f1590096f20 msgr2=0x7f1590097370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.763+0000 7f159589c640 1 --2- 172.21.15.47:0/2136759661 >> v2:172.21.15.47:3300/0 conn(0x7f1590096f20 0x7f1590097370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.763+0000 7f159589c640 1 -- 172.21.15.47:0/2136759661 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590099db0 con 0x7f1590129c70 2024-09-15T21:55:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.763+0000 7f0a777fe640 1 -- 172.21.15.47:0/2036932342 >> v1:172.21.15.47:6789/0 conn(0x7f0a78090d70 legacy=0x7f0a780950e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.763+0000 7f0a777fe640 1 -- 172.21.15.47:0/2036932342 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7808f3c0 con 0x7f0a780957f0 2024-09-15T21:55:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.763+0000 7f159609d640 1 --2- 172.21.15.47:0/2136759661 >> v2:172.21.15.47:3300/0 conn(0x7f1590096f20 0x7f1590097370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:55:22.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.763+0000 7f159589c640 1 --2- 172.21.15.47:0/2136759661 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590129c70 0x7f159012a040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f158c009910 tx=0x7f158c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10c4c3ef896109a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.763+0000 7f0a777fe640 1 --2- 172.21.15.47:0/2036932342 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a780957f0 0x7f0a78095bf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00bb20 tx=0x7f0a6c00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619775cb37109a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.764+0000 7f159489a640 1 -- 172.21.15.47:0/2136759661 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c02ac90 con 0x7f1590129c70 2024-09-15T21:55:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.764+0000 7f0a76ffd640 1 -- 172.21.15.47:0/2036932342 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c031510 con 0x7f0a780957f0 2024-09-15T21:55:22.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.764+0000 7f159489a640 1 -- 172.21.15.47:0/2136759661 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c02adf0 con 0x7f1590129c70 2024-09-15T21:55:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.764+0000 7f0a76ffd640 1 -- 172.21.15.47:0/2036932342 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c02f050 con 0x7f0a780957f0 2024-09-15T21:55:22.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.764+0000 7f159489a640 1 -- 172.21.15.47:0/2136759661 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c032670 con 0x7f1590129c70 2024-09-15T21:55:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.764+0000 7f0a76ffd640 1 -- 172.21.15.47:0/2036932342 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0318a0 con 0x7f0a780957f0 2024-09-15T21:55:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.764+0000 7f0a7e31d640 1 -- 172.21.15.47:0/2036932342 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a780957f0 msgr2=0x7f0a78095bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.764+0000 7f0a7e31d640 1 --2- 172.21.15.47:0/2036932342 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a780957f0 0x7f0a78095bf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00bb20 tx=0x7f0a6c00b3b0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f0a7e31d640 1 -- 172.21.15.47:0/2036932342 shutdown_connections 2024-09-15T21:55:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f0a7e31d640 1 --2- 172.21.15.47:0/2036932342 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a7812c230 0x7f0a7812e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f0a7e31d640 1 --2- 172.21.15.47:0/2036932342 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a780957f0 0x7f0a78095bf0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f0a7e31d640 1 -- 172.21.15.47:0/2036932342 >> 172.21.15.47:0/2036932342 conn(0x7f0a7809a620 msgr2=0x7f0a7809aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f0a7e31d640 1 -- 172.21.15.47:0/2036932342 shutdown_connections 2024-09-15T21:55:22.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f0a7e31d640 1 -- 172.21.15.47:0/2036932342 wait complete. 2024-09-15T21:55:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f1597b00640 1 -- 172.21.15.47:0/2136759661 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590129c70 msgr2=0x7f159012a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f1597b00640 1 --2- 172.21.15.47:0/2136759661 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590129c70 0x7f159012a040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f158c009910 tx=0x7f158c0091d0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f1597b00640 1 -- 172.21.15.47:0/2136759661 shutdown_connections 2024-09-15T21:55:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f1597b00640 1 --2- 172.21.15.47:0/2136759661 >> v2:172.21.15.47:3300/0 conn(0x7f1590096f20 0x7f1590097370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f1597b00640 1 --2- 172.21.15.47:0/2136759661 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 0x7f15900969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f1597b00640 1 --2- 172.21.15.47:0/2136759661 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590129c70 0x7f159012a040 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f1597b00640 1 -- 172.21.15.47:0/2136759661 >> 172.21.15.47:0/2136759661 conn(0x7f159009d850 msgr2=0x7f159009dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f1597b00640 1 -- 172.21.15.47:0/2136759661 shutdown_connections 2024-09-15T21:55:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.765+0000 7f1597b00640 1 -- 172.21.15.47:0/2136759661 wait complete. 2024-09-15T21:55:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f0a7e31d640 1 Processor -- start 2024-09-15T21:55:22.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f0a7e31d640 1 -- start start 2024-09-15T21:55:22.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f1597b00640 1 Processor -- start 2024-09-15T21:55:22.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f1597b00640 1 -- start start 2024-09-15T21:55:22.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f0a7e31d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a78090d70 0x7f0a7812bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f0a7e31d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a780957f0 0x7f0a78089c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f0a7e31d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a7812c230 0x7f0a7808a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f0a7e31d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7808ce10 con 0x7f0a78090d70 2024-09-15T21:55:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f0a7e31d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7812c090 con 0x7f0a780957f0 2024-09-15T21:55:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f0a7e31d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7808d1e0 con 0x7f0a7812c230 2024-09-15T21:55:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.766+0000 7f0a777fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a780957f0 0x7f0a78089c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a777fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a780957f0 0x7f0a78089c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52446/0 (socket says 172.21.15.47:52446) 2024-09-15T21:55:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a777fe640 1 -- 172.21.15.47:0/22205832 learned_addr learned my addr 172.21.15.47:0/22205832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a7c8ba640 1 --2- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a7812c230 0x7f0a7808a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a77fff640 1 --2- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a78090d70 0x7f0a7812bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f1597b00640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590096f20 0x7f1590128700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a777fe640 1 -- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a7812c230 msgr2=0x7f0a7808a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a777fe640 1 --2- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a7812c230 0x7f0a7808a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a777fe640 1 -- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a78090d70 msgr2=0x7f0a7812bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a777fe640 1 --2- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a78090d70 0x7f0a7812bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a777fe640 1 -- 172.21.15.47:0/22205832 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c00b630 con 0x7f0a780957f0 2024-09-15T21:55:22.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a7c8ba640 1 --2- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a7812c230 0x7f0a7808a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f1597b00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1590129c70 0x7f1590128c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f1597b00640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 0x7f159011b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f159589c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590096f20 0x7f1590128700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f159589c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590096f20 0x7f1590128700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52448/0 (socket says 172.21.15.47:52448) 2024-09-15T21:55:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f159509b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1590129c70 0x7f1590128c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f159509b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1590129c70 0x7f1590128c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47906/0 (socket says 172.21.15.47:47906) 2024-09-15T21:55:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f0a777fe640 1 --2- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a780957f0 0x7f0a78089c60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00b600 tx=0x7f0a6c039000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f159609d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 0x7f159011b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f159609d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 0x7f159011b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48150/0 (socket says 172.21.15.47:48150) 2024-09-15T21:55:22.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.767+0000 7f1597b00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159011f260 con 0x7f1590129c70 2024-09-15T21:55:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f1597b00640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1590129180 con 0x7f1590096f20 2024-09-15T21:55:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f159589c640 1 -- 172.21.15.47:0/3274441118 learned_addr learned my addr 172.21.15.47:0/3274441118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f159011da70 con 0x7f159012a580 2024-09-15T21:55:22.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f0a5ffff640 1 -- 172.21.15.47:0/22205832 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c033ac0 con 0x7f0a780957f0 2024-09-15T21:55:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f0a5ffff640 1 -- 172.21.15.47:0/22205832 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c033c20 con 0x7f0a780957f0 2024-09-15T21:55:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f0a5ffff640 1 -- 172.21.15.47:0/22205832 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c03d440 con 0x7f0a780957f0 2024-09-15T21:55:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f0a7e31d640 1 -- 172.21.15.47:0/22205832 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7808c630 con 0x7f0a780957f0 2024-09-15T21:55:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f0a7e31d640 1 -- 172.21.15.47:0/22205832 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7808a6e0 con 0x7f0a780957f0 2024-09-15T21:55:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f159589c640 1 -- 172.21.15.47:0/3274441118 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 msgr2=0x7f159011b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f159589c640 1 --2- 172.21.15.47:0/3274441118 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 0x7f159011b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f159589c640 1 -- 172.21.15.47:0/3274441118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1590129c70 msgr2=0x7f1590128c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f159589c640 1 --2- 172.21.15.47:0/3274441118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1590129c70 0x7f1590128c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.768+0000 7f159589c640 1 -- 172.21.15.47:0/3274441118 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c009450 con 0x7f1590096f20 2024-09-15T21:55:22.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.769+0000 7f159589c640 1 --2- 172.21.15.47:0/3274441118 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590096f20 0x7f1590128700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f158c02a5c0 tx=0x7f158c032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:22.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.769+0000 7f1585ffb640 1 -- 172.21.15.47:0/3274441118 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c03d630 con 0x7f1590096f20 2024-09-15T21:55:22.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.769+0000 7f1585ffb640 1 -- 172.21.15.47:0/3274441118 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c03d7b0 con 0x7f1590096f20 2024-09-15T21:55:22.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.769+0000 7f1585ffb640 1 -- 172.21.15.47:0/3274441118 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c03db70 con 0x7f1590096f20 2024-09-15T21:55:22.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.769+0000 7f0a5ffff640 1 -- 172.21.15.47:0/22205832 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f0a6c03d850 con 0x7f0a780957f0 2024-09-15T21:55:22.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.769+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159011e970 con 0x7f1590096f20 2024-09-15T21:55:22.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.770+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159011beb0 con 0x7f1590096f20 2024-09-15T21:55:22.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.770+0000 7f0a5ffff640 1 -- 172.21.15.47:0/22205832 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f0a6c084160 con 0x7f0a780957f0 2024-09-15T21:55:22.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.770+0000 7f0a7e31d640 1 -- 172.21.15.47:0/22205832 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7808b500 con 0x7f0a780957f0 2024-09-15T21:55:22.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.771+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159011ef40 con 0x7f1590096f20 2024-09-15T21:55:22.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.771+0000 7f1585ffb640 1 -- 172.21.15.47:0/3274441118 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f158c039040 con 0x7f1590096f20 2024-09-15T21:55:22.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.777+0000 7f1585ffb640 1 -- 172.21.15.47:0/3274441118 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f158c061860 con 0x7f1590096f20 2024-09-15T21:55:22.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.777+0000 7f0a5ffff640 1 -- 172.21.15.47:0/22205832 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f0a6c041020 con 0x7f0a780957f0 2024-09-15T21:55:22.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.784+0000 7f1585ffb640 1 -- 172.21.15.47:0/3274441118 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f159011ef40 con 0x7f1590096f20 2024-09-15T21:55:22.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.802+0000 7fceb8f9d640 1 Processor -- start 2024-09-15T21:55:22.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.802+0000 7fceb8f9d640 1 -- start start 2024-09-15T21:55:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.803+0000 7fceb8f9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fceb4096150 0x7fceb4096520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.803+0000 7fceb8f9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb412d260 0x7fceb412f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.803+0000 7fceb8f9d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb40907c0 con 0x7fceb4096a60 2024-09-15T21:55:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.803+0000 7fceb8f9d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb40900a0 con 0x7fceb4096150 2024-09-15T21:55:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.803+0000 7fceb8f9d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fceb408fa90 con 0x7fceb412d260 2024-09-15T21:55:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.803+0000 7fceb259c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fceb4096150 0x7fceb4096520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.803+0000 7fceb2d9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb412d260 0x7fceb412f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.803+0000 7fceb1d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fceb4096a60 0x7fceb4096e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53046/0 (socket says 172.21.15.47:53046) 2024-09-15T21:55:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.803+0000 7fceb1d9b640 1 -- 172.21.15.47:0/3574128353 learned_addr learned my addr 172.21.15.47:0/3574128353 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:22.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.804+0000 7fceb259c640 1 -- 172.21.15.47:0/3574128353 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb412d260 msgr2=0x7fceb412f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.804+0000 7fceb259c640 1 --2- 172.21.15.47:0/3574128353 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb412d260 0x7fceb412f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.804+0000 7fceb259c640 1 -- 172.21.15.47:0/3574128353 >> v1:172.21.15.47:6789/0 conn(0x7fceb4096a60 legacy=0x7fceb4096e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.804+0000 7fceb159a640 1 -- 172.21.15.47:0/3574128353 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771887485 0 0) 0x7fceb40907c0 con 0x7fceb4096a60 2024-09-15T21:55:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.804+0000 7fceb259c640 1 -- 172.21.15.47:0/3574128353 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb4091020 con 0x7fceb4096150 2024-09-15T21:55:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.805+0000 7fceb259c640 1 --2- 172.21.15.47:0/3574128353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb4096150 0x7fceb4096520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009e60 tx=0x7fcea4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8049bfa9526730fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:22.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.805+0000 7fceb159a640 1 -- 172.21.15.47:0/3574128353 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea40306b0 con 0x7fceb4096150 2024-09-15T21:55:22.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.805+0000 7fceb159a640 1 -- 172.21.15.47:0/3574128353 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea4030810 con 0x7fceb4096150 2024-09-15T21:55:22.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.805+0000 7fceb159a640 1 -- 172.21.15.47:0/3574128353 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea4030bb0 con 0x7fceb4096150 2024-09-15T21:55:22.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.806+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3574128353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb4096150 msgr2=0x7fceb4096520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.806+0000 7fceb8f9d640 1 --2- 172.21.15.47:0/3574128353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb4096150 0x7fceb4096520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009e60 tx=0x7fcea4009780 comp rx=0 tx=0).stop 2024-09-15T21:55:22.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.806+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3574128353 shutdown_connections 2024-09-15T21:55:22.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.806+0000 7fceb8f9d640 1 --2- 172.21.15.47:0/3574128353 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb412d260 0x7fceb412f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.806+0000 7fceb8f9d640 1 --2- 172.21.15.47:0/3574128353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb4096150 0x7fceb4096520 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.806+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3574128353 >> 172.21.15.47:0/3574128353 conn(0x7fceb409b450 msgr2=0x7fceb409b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:22.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.806+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3574128353 shutdown_connections 2024-09-15T21:55:22.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.806+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3574128353 wait complete. 2024-09-15T21:55:22.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.807+0000 7fceb8f9d640 1 Processor -- start 2024-09-15T21:55:22.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.807+0000 7fa6c219d640 1 Processor -- start 2024-09-15T21:55:22.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.807+0000 7fceb8f9d640 1 -- start start 2024-09-15T21:55:22.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.807+0000 7fa6c219d640 1 -- start start 2024-09-15T21:55:22.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb8f9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb4096150 0x7fceb412c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb8f9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fceb4096a60 0x7fceb412c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb8f9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb412d260 0x7fceb412ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb259c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb4096150 0x7fceb412c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb259c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb4096150 0x7fceb412c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48178/0 (socket says 172.21.15.47:48178) 2024-09-15T21:55:22.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fa6c219d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa6bc095520 0x7fa6bc06a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb412d260 0x7fceb412ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb2d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb412d260 0x7fceb412ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47928/0 (socket says 172.21.15.47:47928) 2024-09-15T21:55:22.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb259c640 1 -- 172.21.15.47:0/3691056000 learned_addr learned my addr 172.21.15.47:0/3691056000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:22.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fa6c219d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa6bc06a5c0 0x7fa6bc06a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb1d9b640 1 --2- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fceb4096a60 0x7fceb412c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb408d090 con 0x7fceb412d260 2024-09-15T21:55:22.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb408ecd0 con 0x7fceb4096a60 2024-09-15T21:55:22.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fceb408ccc0 con 0x7fceb4096150 2024-09-15T21:55:22.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fa6baffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa6bc06a5c0 0x7fa6bc06a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fa6baffd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa6bc06a5c0 0x7fa6bc06a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47938/0 (socket says 172.21.15.47:47938) 2024-09-15T21:55:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fa6bbfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa6bc12ac90 0x7fa6bc12d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53052/0 (socket says 172.21.15.47:53052) 2024-09-15T21:55:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fa6baffd640 1 -- 172.21.15.47:0/2745893015 learned_addr learned my addr 172.21.15.47:0/2745893015 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fa6c219d640 1 -- 172.21.15.47:0/2745893015 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6bc093c10 con 0x7fa6bc12ac90 2024-09-15T21:55:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fa6c219d640 1 -- 172.21.15.47:0/2745893015 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6bc0930c0 con 0x7fa6bc06a5c0 2024-09-15T21:55:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.808+0000 7fa6c219d640 1 -- 172.21.15.47:0/2745893015 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc06aef0 con 0x7fa6bc095520 2024-09-15T21:55:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fceb2d9d640 1 -- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb4096150 msgr2=0x7fceb412c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fceb2d9d640 1 --2- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb4096150 0x7fceb412c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fceb2d9d640 1 -- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fceb4096a60 msgr2=0x7fceb412c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fceb2d9d640 1 --2- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fceb4096a60 0x7fceb412c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fceb2d9d640 1 -- 172.21.15.47:0/3691056000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea4009a20 con 0x7fceb412d260 2024-09-15T21:55:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fa6baffd640 1 -- 172.21.15.47:0/2745893015 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa6bc095520 msgr2=0x7fa6bc06a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fa6baffd640 1 --2- 172.21.15.47:0/2745893015 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa6bc095520 0x7fa6bc06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fa6baffd640 1 -- 172.21.15.47:0/2745893015 >> v1:172.21.15.47:6789/0 conn(0x7fa6bc12ac90 legacy=0x7fa6bc12d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fa6baffd640 1 -- 172.21.15.47:0/2745893015 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc093fd0 con 0x7fa6bc06a5c0 2024-09-15T21:55:22.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fceb2d9d640 1 --2- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb412d260 0x7fceb412ccf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fceb408f750 tx=0x7fceac0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:22.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fcea27fc640 1 -- 172.21.15.47:0/3691056000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac009520 con 0x7fceb412d260 2024-09-15T21:55:22.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fcea27fc640 1 -- 172.21.15.47:0/3691056000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceac00c940 con 0x7fceb412d260 2024-09-15T21:55:22.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.809+0000 7fcea27fc640 1 -- 172.21.15.47:0/3691056000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac00cd00 con 0x7fceb412d260 2024-09-15T21:55:22.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.810+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb408c4e0 con 0x7fceb412d260 2024-09-15T21:55:22.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.810+0000 7fa6baffd640 1 --2- 172.21.15.47:0/2745893015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc06a5c0 0x7fa6bc06a9b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000aac0 tx=0x7fa6b000a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0ddcbc27f96631 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.810+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb4089a90 con 0x7fceb412d260 2024-09-15T21:55:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.810+0000 7fa6ba7fc640 1 -- 172.21.15.47:0/2745893015 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0030510 con 0x7fa6bc06a5c0 2024-09-15T21:55:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.810+0000 7fa6ba7fc640 1 -- 172.21.15.47:0/2745893015 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b002e050 con 0x7fa6bc06a5c0 2024-09-15T21:55:22.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.810+0000 7fa6ba7fc640 1 -- 172.21.15.47:0/2745893015 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00308e0 con 0x7fa6bc06a5c0 2024-09-15T21:55:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.811+0000 7fcea27fc640 1 -- 172.21.15.47:0/3691056000 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fceac013680 con 0x7fceb412d260 2024-09-15T21:55:22.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.811+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb404bf60 con 0x7fceb412d260 2024-09-15T21:55:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.811+0000 7fa6c219d640 1 -- 172.21.15.47:0/2745893015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc06a5c0 msgr2=0x7fa6bc06a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.811+0000 7fa6c219d640 1 --2- 172.21.15.47:0/2745893015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc06a5c0 0x7fa6bc06a9b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000aac0 tx=0x7fa6b000a360 comp rx=0 tx=0).stop 2024-09-15T21:55:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.811+0000 7fa6c219d640 1 -- 172.21.15.47:0/2745893015 shutdown_connections 2024-09-15T21:55:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.811+0000 7fa6c219d640 1 --2- 172.21.15.47:0/2745893015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc06a5c0 0x7fa6bc06a9b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.811+0000 7fa6c219d640 1 --2- 172.21.15.47:0/2745893015 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa6bc095520 0x7fa6bc06a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.811+0000 7fa6c219d640 1 -- 172.21.15.47:0/2745893015 >> 172.21.15.47:0/2745893015 conn(0x7fa6bc096610 msgr2=0x7fa6bc096a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.811+0000 7fa6c219d640 1 -- 172.21.15.47:0/2745893015 shutdown_connections 2024-09-15T21:55:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.812+0000 7fa6c219d640 1 -- 172.21.15.47:0/2745893015 wait complete. 2024-09-15T21:55:22.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.812+0000 7fcea27fc640 1 -- 172.21.15.47:0/3691056000 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fceac063460 con 0x7fceb412d260 2024-09-15T21:55:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6c219d640 1 Processor -- start 2024-09-15T21:55:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6c219d640 1 -- start start 2024-09-15T21:55:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6c219d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa6bc06a5c0 0x7fa6bc129fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6c219d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6bc095520 0x7fa6bc12a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6c219d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc12ac90 0x7fa6bc08f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6c219d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc090af0 con 0x7fa6bc12ac90 2024-09-15T21:55:22.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6bb7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa6bc06a5c0 0x7fa6bc129fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6c219d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc092780 con 0x7fa6bc095520 2024-09-15T21:55:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6bb7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa6bc06a5c0 0x7fa6bc129fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48204/0 (socket says 172.21.15.47:48204) 2024-09-15T21:55:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6bb7fe640 1 -- 172.21.15.47:0/2100424498 learned_addr learned my addr 172.21.15.47:0/2100424498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc092a40 con 0x7fa6bc06a5c0 2024-09-15T21:55:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.813+0000 7fa6bbfff640 1 --2- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc12ac90 0x7fa6bc08f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.814+0000 7fa6bbfff640 1 -- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa6bc06a5c0 msgr2=0x7fa6bc129fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.814+0000 7fa6bbfff640 1 --2- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa6bc06a5c0 0x7fa6bc129fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.814+0000 7fa6bbfff640 1 -- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6bc095520 msgr2=0x7fa6bc12a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.814+0000 7fa6bbfff640 1 --2- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6bc095520 0x7fa6bc12a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.814+0000 7fa6bbfff640 1 -- 172.21.15.47:0/2100424498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b000a600 con 0x7fa6bc12ac90 2024-09-15T21:55:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.814+0000 7fa6bbfff640 1 --2- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc12ac90 0x7fa6bc08f1f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400c790 tx=0x7fa6b40015e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:22.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.818+0000 7fcea27fc640 1 -- 172.21.15.47:0/3691056000 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fceac03e520 con 0x7fceb412d260 2024-09-15T21:55:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.819+0000 7fa6a37fe640 1 -- 172.21.15.47:0/2100424498 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4004b20 con 0x7fa6bc12ac90 2024-09-15T21:55:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.819+0000 7fa6a37fe640 1 -- 172.21.15.47:0/2100424498 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b4004c80 con 0x7fa6bc12ac90 2024-09-15T21:55:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.819+0000 7fa6a37fe640 1 -- 172.21.15.47:0/2100424498 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400a890 con 0x7fa6bc12ac90 2024-09-15T21:55:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.819+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc092470 con 0x7fa6bc12ac90 2024-09-15T21:55:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.819+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc091430 con 0x7fa6bc12ac90 2024-09-15T21:55:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.820+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc04bf60 con 0x7fa6bc12ac90 2024-09-15T21:55:22.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.820+0000 7fa6a37fe640 1 -- 172.21.15.47:0/2100424498 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fa6b400acc0 con 0x7fa6bc12ac90 2024-09-15T21:55:22.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.821+0000 7fa6a37fe640 1 -- 172.21.15.47:0/2100424498 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fa6b4062850 con 0x7fa6bc12ac90 2024-09-15T21:55:22.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.828+0000 7fa6a37fe640 1 -- 172.21.15.47:0/2100424498 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fa6b403da70 con 0x7fa6bc12ac90 2024-09-15T21:55:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.936+0000 7f0a7e31d640 1 -- 172.21.15.47:0/22205832 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0a7804bf60 con 0x7f0a780957f0 2024-09-15T21:55:22.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.937+0000 7f0a5ffff640 1 -- 172.21.15.47:0/22205832 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a6c063130 con 0x7f0a780957f0 2024-09-15T21:55:22.939 INFO:teuthology.orchestra.run.smithi047.stdout:38654705667 2024-09-15T21:55:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.937+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f159011ef40 con 0x7f1590096f20 2024-09-15T21:55:22.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.938+0000 7f1585ffb640 1 -- 172.21.15.47:0/3274441118 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f159011ef40 con 0x7f1590096f20 2024-09-15T21:55:22.940 INFO:teuthology.orchestra.run.smithi047.stdout:38654705666 2024-09-15T21:55:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f0a7e31d640 1 -- 172.21.15.47:0/22205832 >> v1:172.21.15.47:6800/32267 conn(0x7f0a6404cc70 legacy=0x7f0a6404f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f0a7e31d640 1 -- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a780957f0 msgr2=0x7f0a78089c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f0a7e31d640 1 --2- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a780957f0 0x7f0a78089c60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00b600 tx=0x7f0a6c039000 comp rx=0 tx=0).stop 2024-09-15T21:55:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f0a7e31d640 1 -- 172.21.15.47:0/22205832 shutdown_connections 2024-09-15T21:55:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f0a7e31d640 1 --2- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0a7812c230 0x7f0a7808a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 >> v1:172.21.15.47:6800/32267 conn(0x7f157404cba0 legacy=0x7f157404f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590096f20 msgr2=0x7f1590128700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f1597b00640 1 --2- 172.21.15.47:0/3274441118 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590096f20 0x7f1590128700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f158c02a5c0 tx=0x7f158c032000 comp rx=0 tx=0).stop 2024-09-15T21:55:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f0a7e31d640 1 --2- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f0a780957f0 0x7f0a78089c60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f0a7e31d640 1 --2- 172.21.15.47:0/22205832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a78090d70 0x7f0a7812bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f0a7e31d640 1 -- 172.21.15.47:0/22205832 >> 172.21.15.47:0/22205832 conn(0x7f0a7809a620 msgr2=0x7f0a78098a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 shutdown_connections 2024-09-15T21:55:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f1597b00640 1 --2- 172.21.15.47:0/3274441118 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f159012a580 0x7f159011b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f1597b00640 1 --2- 172.21.15.47:0/3274441118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1590129c70 0x7f1590128c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.939+0000 7f1597b00640 1 --2- 172.21.15.47:0/3274441118 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1590096f20 0x7f1590128700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.940+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 >> 172.21.15.47:0/3274441118 conn(0x7f159009d850 msgr2=0x7f1590114700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.940+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 shutdown_connections 2024-09-15T21:55:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.940+0000 7f1597b00640 1 -- 172.21.15.47:0/3274441118 wait complete. 2024-09-15T21:55:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.940+0000 7f0a7e31d640 1 -- 172.21.15.47:0/22205832 shutdown_connections 2024-09-15T21:55:22.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.940+0000 7f0a7e31d640 1 -- 172.21.15.47:0/22205832 wait complete. 2024-09-15T21:55:22.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705666 for osd.0 2024-09-15T21:55:22.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705667 for osd.3 2024-09-15T21:55:22.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.959+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fceb408e3e0 con 0x7fceb412d260 2024-09-15T21:55:22.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.960+0000 7fcea27fc640 1 -- 172.21.15.47:0/3691056000 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fceb408e3e0 con 0x7fceb412d260 2024-09-15T21:55:22.962 INFO:teuthology.orchestra.run.smithi047.stdout:38654705666 2024-09-15T21:55:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.962+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 >> v1:172.21.15.47:6800/32267 conn(0x7fce9404cb70 legacy=0x7fce9404f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.962+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb412d260 msgr2=0x7fceb412ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.962+0000 7fceb8f9d640 1 --2- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb412d260 0x7fceb412ccf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fceb408f750 tx=0x7fceac0090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.962+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 shutdown_connections 2024-09-15T21:55:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.962+0000 7fceb8f9d640 1 --2- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fceb412d260 0x7fceb412ccf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.962+0000 7fceb8f9d640 1 --2- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fceb4096a60 0x7fceb412c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.962+0000 7fceb8f9d640 1 --2- 172.21.15.47:0/3691056000 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fceb4096150 0x7fceb412c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.962+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 >> 172.21.15.47:0/3691056000 conn(0x7fceb409b450 msgr2=0x7fceb4094f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.962+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 shutdown_connections 2024-09-15T21:55:22.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.962+0000 7fceb8f9d640 1 -- 172.21.15.47:0/3691056000 wait complete. 2024-09-15T21:55:22.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705666 for osd.1 2024-09-15T21:55:22.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.977+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa6bc06f0d0 con 0x7fa6bc12ac90 2024-09-15T21:55:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.977+0000 7fa6a37fe640 1 -- 172.21.15.47:0/2100424498 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa6b4018030 con 0x7fa6bc12ac90 2024-09-15T21:55:22.980 INFO:teuthology.orchestra.run.smithi047.stdout:38654705667 2024-09-15T21:55:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.979+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 >> v1:172.21.15.47:6800/32267 conn(0x7fa6a804cb70 legacy=0x7fa6a804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.979+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc12ac90 msgr2=0x7fa6bc08f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.979+0000 7fa6c219d640 1 --2- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc12ac90 0x7fa6bc08f1f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400c790 tx=0x7fa6b40015e0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.979+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 shutdown_connections 2024-09-15T21:55:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.979+0000 7fa6c219d640 1 --2- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6bc12ac90 0x7fa6bc08f1f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.979+0000 7fa6c219d640 1 --2- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa6bc095520 0x7fa6bc12a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.979+0000 7fa6c219d640 1 --2- 172.21.15.47:0/2100424498 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa6bc06a5c0 0x7fa6bc129fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.979+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 >> 172.21.15.47:0/2100424498 conn(0x7fa6bc096610 msgr2=0x7fa6bc087f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:22.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.979+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 shutdown_connections 2024-09-15T21:55:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:22.979+0000 7fa6c219d640 1 -- 172.21.15.47:0/2100424498 wait complete. 2024-09-15T21:55:22.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705667 for osd.2 2024-09-15T21:55:23.954 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:55:23.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:55:23.975 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:55:23.993 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:55:24.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.066+0000 7f811e75d640 1 Processor -- start 2024-09-15T21:55:24.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.066+0000 7f811e75d640 1 -- start start 2024-09-15T21:55:24.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.066+0000 7f811e75d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 0x7f8118127d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.066+0000 7f811e75d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118088680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.066+0000 7f811e75d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8118127ad0 con 0x7f8118128240 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.066+0000 7f811e75d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811808aad0 con 0x7f811806eec0 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.066+0000 7f811e75d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f811808df60 con 0x7f8118128670 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.066+0000 7f811ccfa640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118088680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.067+0000 7f81177fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8118128240 0x7f8118087f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53062/0 (socket says 172.21.15.47:53062) 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.067+0000 7f81177fe640 1 -- 172.21.15.47:0/1500789069 learned_addr learned my addr 172.21.15.47:0/1500789069 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.067+0000 7f811ccfa640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118088680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48218/0 (socket says 172.21.15.47:48218) 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.067+0000 7f8117fff640 1 --2- 172.21.15.47:0/1500789069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 0x7f8118127d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.067+0000 7f811ccfa640 1 -- 172.21.15.47:0/1500789069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 msgr2=0x7f8118127d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.067+0000 7f811ccfa640 1 --2- 172.21.15.47:0/1500789069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 0x7f8118127d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.067+0000 7f811ccfa640 1 -- 172.21.15.47:0/1500789069 >> v1:172.21.15.47:6789/0 conn(0x7f8118128240 legacy=0x7f8118087f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.067+0000 7f811ccfa640 1 -- 172.21.15.47:0/1500789069 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8118089ec0 con 0x7f8118128670 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.068+0000 7f811ccfa640 1 --2- 172.21.15.47:0/1500789069 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118088680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8110009e90 tx=0x7f8110009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed233767118b7e27 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.068+0000 7f8116ffd640 1 -- 172.21.15.47:0/1500789069 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110007550 con 0x7f8118128670 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.068+0000 7f8116ffd640 1 -- 172.21.15.47:0/1500789069 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811002d050 con 0x7f8118128670 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.068+0000 7f8116ffd640 1 -- 172.21.15.47:0/1500789069 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110007920 con 0x7f8118128670 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.069+0000 7f811e75d640 1 -- 172.21.15.47:0/1500789069 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 msgr2=0x7f8118088680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.069+0000 7f811e75d640 1 --2- 172.21.15.47:0/1500789069 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118088680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8110009e90 tx=0x7f8110009780 comp rx=0 tx=0).stop 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.069+0000 7f811e75d640 1 -- 172.21.15.47:0/1500789069 shutdown_connections 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.069+0000 7f811e75d640 1 --2- 172.21.15.47:0/1500789069 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118088680 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.069+0000 7f811e75d640 1 --2- 172.21.15.47:0/1500789069 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 0x7f8118127d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.069+0000 7f811e75d640 1 -- 172.21.15.47:0/1500789069 >> 172.21.15.47:0/1500789069 conn(0x7f811809b700 msgr2=0x7f811809bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.069+0000 7f811e75d640 1 -- 172.21.15.47:0/1500789069 shutdown_connections 2024-09-15T21:55:24.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.069+0000 7f811e75d640 1 -- 172.21.15.47:0/1500789069 wait complete. 2024-09-15T21:55:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.070+0000 7f811e75d640 1 Processor -- start 2024-09-15T21:55:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.070+0000 7f811e75d640 1 -- start start 2024-09-15T21:55:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.070+0000 7f811e75d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 0x7f8118137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.070+0000 7f811e75d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8118128240 0x7f8118137740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.070+0000 7f811e75d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118137c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f811e75d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811812b6b0 con 0x7f8118128240 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f811e75d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811808d260 con 0x7f811806eec0 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f811e75d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f811812b270 con 0x7f8118128670 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f8117fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 0x7f8118137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f811ccfa640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118137c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f81177fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8118128240 0x7f8118137740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f811ccfa640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118137c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48232/0 (socket says 172.21.15.47:48232) 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f811ccfa640 1 -- 172.21.15.47:0/3258416272 learned_addr learned my addr 172.21.15.47:0/3258416272 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f8117fff640 1 -- 172.21.15.47:0/3258416272 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 msgr2=0x7f8118137c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f8117fff640 1 --2- 172.21.15.47:0/3258416272 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118137c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f8117fff640 1 -- 172.21.15.47:0/3258416272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8118128240 msgr2=0x7f8118137740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f8117fff640 1 --2- 172.21.15.47:0/3258416272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8118128240 0x7f8118137740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f8117fff640 1 -- 172.21.15.47:0/3258416272 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110009a20 con 0x7f811806eec0 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.071+0000 7f8117fff640 1 --2- 172.21.15.47:0/3258416272 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 0x7f8118137200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f810800e960 tx=0x7f81080090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.072+0000 7f80fffff640 1 -- 172.21.15.47:0/3258416272 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810800c940 con 0x7f811806eec0 2024-09-15T21:55:24.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.072+0000 7f80fffff640 1 -- 172.21.15.47:0/3258416272 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810800caa0 con 0x7f811806eec0 2024-09-15T21:55:24.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.072+0000 7f811e75d640 1 -- 172.21.15.47:0/3258416272 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8118128c60 con 0x7f811806eec0 2024-09-15T21:55:24.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.072+0000 7f80fffff640 1 -- 172.21.15.47:0/3258416272 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810800ce20 con 0x7f811806eec0 2024-09-15T21:55:24.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.072+0000 7f811e75d640 1 -- 172.21.15.47:0/3258416272 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81181290f0 con 0x7f811806eec0 2024-09-15T21:55:24.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.073+0000 7f811e75d640 1 -- 172.21.15.47:0/3258416272 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811806d7f0 con 0x7f811806eec0 2024-09-15T21:55:24.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.073+0000 7f80fffff640 1 -- 172.21.15.47:0/3258416272 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f8108013680 con 0x7f811806eec0 2024-09-15T21:55:24.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.074+0000 7f80fffff640 1 -- 172.21.15.47:0/3258416272 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f8108063590 con 0x7f811806eec0 2024-09-15T21:55:24.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.077+0000 7f80fffff640 1 -- 172.21.15.47:0/3258416272 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f811806d7f0 con 0x7f811806eec0 2024-09-15T21:55:24.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.105+0000 7fdebc22b640 1 Processor -- start 2024-09-15T21:55:24.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.105+0000 7f79286dd640 1 Processor -- start 2024-09-15T21:55:24.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.105+0000 7f79286dd640 1 -- start start 2024-09-15T21:55:24.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.105+0000 7fdebc22b640 1 -- start start 2024-09-15T21:55:24.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdebc22b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb408e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdebc22b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdeb408ebf0 0x7fdeb412b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdebc22b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb412b710 0x7fdeb412daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdebc22b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb4069830 con 0x7fdeb408ebf0 2024-09-15T21:55:24.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdebc22b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb408f4d0 con 0x7fdeb408e2e0 2024-09-15T21:55:24.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdebc22b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb408f820 con 0x7fdeb412b710 2024-09-15T21:55:24.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7f79286dd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920095c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdeba7c8640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb412b710 0x7fdeb412daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdeb9fc7640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb408e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdeba7c8640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb412b710 0x7fdeb412daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48234/0 (socket says 172.21.15.47:48234) 2024-09-15T21:55:24.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdeba7c8640 1 -- 172.21.15.47:0/940851583 learned_addr learned my addr 172.21.15.47:0/940851583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdeb97c6640 1 --2- 172.21.15.47:0/940851583 >> v2:172.21.15.47:3300/0 conn(0x7fdeb408ebf0 0x7fdeb412b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7f79286dd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920128470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7f79286dd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792006d8c0 con 0x7f792006adb0 2024-09-15T21:55:24.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7f79286dd640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792006a890 con 0x7f7920096160 2024-09-15T21:55:24.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7f79286dd640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f792006d430 con 0x7f7920095850 2024-09-15T21:55:24.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7f7926c7a640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920128470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7f7926c7a640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920128470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52504/0 (socket says 172.21.15.47:52504) 2024-09-15T21:55:24.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7f7926c7a640 1 -- 172.21.15.47:0/356347930 learned_addr learned my addr 172.21.15.47:0/356347930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7f7926479640 1 --2- 172.21.15.47:0/356347930 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920095c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdeb97c6640 1 -- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb412b710 msgr2=0x7fdeb412daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:24.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdeb97c6640 1 --2- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb412b710 0x7fdeb412daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdeb97c6640 1 -- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 msgr2=0x7fdeb408e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:24.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7fdeb97c6640 1 --2- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb408e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.106+0000 7f7926c7a640 1 -- 172.21.15.47:0/356347930 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 msgr2=0x7f7920095c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:24.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.107+0000 7fdeb97c6640 1 -- 172.21.15.47:0/940851583 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb4090770 con 0x7fdeb408ebf0 2024-09-15T21:55:24.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.107+0000 7f7926c7a640 1 --2- 172.21.15.47:0/356347930 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920095c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.107+0000 7f7926c7a640 1 -- 172.21.15.47:0/356347930 >> v1:172.21.15.47:6789/0 conn(0x7f792006adb0 legacy=0x7f7920127d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.107+0000 7f7926c7a640 1 -- 172.21.15.47:0/356347930 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7920095630 con 0x7f7920096160 2024-09-15T21:55:24.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.107+0000 7f7926479640 1 --2- 172.21.15.47:0/356347930 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920095c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:55:24.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.107+0000 7fdeb9fc7640 1 --2- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb408e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:24.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.107+0000 7fdeb97c6640 1 --2- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdeb408ebf0 0x7fdeb412b1d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009880 tx=0x7fdea80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff733f9cd0e1928c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.107+0000 7f7926c7a640 1 --2- 172.21.15.47:0/356347930 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920128470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7918009f50 tx=0x7f7918009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac65172b10167174 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.108+0000 7f7925477640 1 -- 172.21.15.47:0/356347930 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918031400 con 0x7f7920096160 2024-09-15T21:55:24.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.108+0000 7f7925477640 1 -- 172.21.15.47:0/356347930 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791802d050 con 0x7f7920096160 2024-09-15T21:55:24.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.108+0000 7fdeb8fc5640 1 -- 172.21.15.47:0/940851583 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea802abd0 con 0x7fdeb408ebf0 2024-09-15T21:55:24.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.108+0000 7f7925477640 1 -- 172.21.15.47:0/356347930 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180317d0 con 0x7f7920096160 2024-09-15T21:55:24.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.108+0000 7fdeb8fc5640 1 -- 172.21.15.47:0/940851583 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea802ad30 con 0x7fdeb408ebf0 2024-09-15T21:55:24.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.108+0000 7fdeb8fc5640 1 -- 172.21.15.47:0/940851583 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8032670 con 0x7fdeb408ebf0 2024-09-15T21:55:24.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.108+0000 7f79286dd640 1 -- 172.21.15.47:0/356347930 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 msgr2=0x7f7920128470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.108+0000 7f79286dd640 1 --2- 172.21.15.47:0/356347930 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920128470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7918009f50 tx=0x7f7918009840 comp rx=0 tx=0).stop 2024-09-15T21:55:24.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7fdebc22b640 1 -- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdeb408ebf0 msgr2=0x7fdeb412b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7fdebc22b640 1 --2- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdeb408ebf0 0x7fdeb412b1d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009880 tx=0x7fdea80090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7fdebc22b640 1 -- 172.21.15.47:0/940851583 shutdown_connections 2024-09-15T21:55:24.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7fdebc22b640 1 --2- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb412b710 0x7fdeb412daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7fdebc22b640 1 --2- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdeb408ebf0 0x7fdeb412b1d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7fdebc22b640 1 --2- 172.21.15.47:0/940851583 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb408e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7fdebc22b640 1 -- 172.21.15.47:0/940851583 >> 172.21.15.47:0/940851583 conn(0x7fdeb409ac50 msgr2=0x7fdeb4096b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7f79286dd640 1 -- 172.21.15.47:0/356347930 shutdown_connections 2024-09-15T21:55:24.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7f79286dd640 1 --2- 172.21.15.47:0/356347930 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920128470 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7f79286dd640 1 --2- 172.21.15.47:0/356347930 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920095c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7f79286dd640 1 -- 172.21.15.47:0/356347930 >> 172.21.15.47:0/356347930 conn(0x7f7920096610 msgr2=0x7f7920096a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7fdebc22b640 1 -- 172.21.15.47:0/940851583 shutdown_connections 2024-09-15T21:55:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7f79286dd640 1 -- 172.21.15.47:0/356347930 shutdown_connections 2024-09-15T21:55:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7f79286dd640 1 -- 172.21.15.47:0/356347930 wait complete. 2024-09-15T21:55:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.109+0000 7fdebc22b640 1 -- 172.21.15.47:0/940851583 wait complete. 2024-09-15T21:55:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.110+0000 7f79286dd640 1 Processor -- start 2024-09-15T21:55:24.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.110+0000 7f79286dd640 1 -- start start 2024-09-15T21:55:24.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.110+0000 7fdebc22b640 1 Processor -- start 2024-09-15T21:55:24.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.110+0000 7fdebc22b640 1 -- start start 2024-09-15T21:55:24.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.110+0000 7f79286dd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920126e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdebc22b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb4124710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.110+0000 7f79286dd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920127380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.110+0000 7f79286dd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79201278c0 0x7f7920135010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdebc22b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb408ebf0 0x7fdeb4124c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdebc22b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdeb412b710 0x7fdeb4125190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdebc22b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4120390 con 0x7fdeb412b710 2024-09-15T21:55:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdebc22b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb41257c0 con 0x7fdeb408e2e0 2024-09-15T21:55:24.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdebc22b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4120bd0 con 0x7fdeb408ebf0 2024-09-15T21:55:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.110+0000 7f79286dd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920128050 con 0x7f79201278c0 2024-09-15T21:55:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.110+0000 7f79286dd640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7920093c70 con 0x7f7920096160 2024-09-15T21:55:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdeb9fc7640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb4124710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdeb97c6640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb408ebf0 0x7fdeb4124c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdeb9fc7640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb4124710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52524/0 (socket says 172.21.15.47:52524) 2024-09-15T21:55:24.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdeb9fc7640 1 -- 172.21.15.47:0/2049491447 learned_addr learned my addr 172.21.15.47:0/2049491447 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.110+0000 7f79286dd640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79200995b0 con 0x7f7920095850 2024-09-15T21:55:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7f7926479640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920126e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7f7926479640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920126e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48254/0 (socket says 172.21.15.47:48254) 2024-09-15T21:55:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7f7926479640 1 -- 172.21.15.47:0/1098625444 learned_addr learned my addr 172.21.15.47:0/1098625444 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7f7926c7a640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79201278c0 0x7f7920135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7f7925c78640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920127380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7f7926c7a640 1 -- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 msgr2=0x7f7920126e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7f7926c7a640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920126e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7f7926c7a640 1 -- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 msgr2=0x7f7920127380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7f7926c7a640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920127380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdeb9fc7640 1 -- 172.21.15.47:0/2049491447 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb408ebf0 msgr2=0x7fdeb4124c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdeb9fc7640 1 --2- 172.21.15.47:0/2049491447 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb408ebf0 0x7fdeb4124c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdeb9fc7640 1 -- 172.21.15.47:0/2049491447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdeb412b710 msgr2=0x7fdeb4125190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:55:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdeb9fc7640 1 --2- 172.21.15.47:0/2049491447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdeb412b710 0x7fdeb4125190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.111+0000 7fdeb9fc7640 1 -- 172.21.15.47:0/2049491447 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac00de30 con 0x7fdeb408e2e0 2024-09-15T21:55:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7f7926c7a640 1 -- 172.21.15.47:0/1098625444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7918009ae0 con 0x7f79201278c0 2024-09-15T21:55:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7f7925c78640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920127380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:55:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7f7926479640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920126e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7fdeb9fc7640 1 --2- 172.21.15.47:0/2049491447 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb4124710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0124f0 tx=0x7fdeac00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7f7926c7a640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79201278c0 0x7f7920135010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f792006da50 tx=0x7f7918036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7fdeb1ffb640 1 -- 172.21.15.47:0/2049491447 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0175d0 con 0x7fdeb408e2e0 2024-09-15T21:55:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7fdeb1ffb640 1 -- 172.21.15.47:0/2049491447 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac013040 con 0x7fdeb408e2e0 2024-09-15T21:55:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7fdeb1ffb640 1 -- 172.21.15.47:0/2049491447 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac017960 con 0x7fdeb408e2e0 2024-09-15T21:55:24.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7fdebc22b640 1 -- 172.21.15.47:0/2049491447 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea8009390 con 0x7fdeb408e2e0 2024-09-15T21:55:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7f79167fc640 1 -- 172.21.15.47:0/1098625444 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918007170 con 0x7f79201278c0 2024-09-15T21:55:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7f79286dd640 1 -- 172.21.15.47:0/1098625444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7920089880 con 0x7f79201278c0 2024-09-15T21:55:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.113+0000 7fdebc22b640 1 -- 172.21.15.47:0/2049491447 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb4134df0 con 0x7fdeb408e2e0 2024-09-15T21:55:24.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7f79167fc640 1 -- 172.21.15.47:0/1098625444 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7918007170 con 0x7f79201278c0 2024-09-15T21:55:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.112+0000 7f79286dd640 1 -- 172.21.15.47:0/1098625444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7920135550 con 0x7f79201278c0 2024-09-15T21:55:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.113+0000 7f79167fc640 1 -- 172.21.15.47:0/1098625444 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918009520 con 0x7f79201278c0 2024-09-15T21:55:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.114+0000 7f79167fc640 1 -- 172.21.15.47:0/1098625444 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f791803c630 con 0x7f79201278c0 2024-09-15T21:55:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.114+0000 7f79286dd640 1 -- 172.21.15.47:0/1098625444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7920128640 con 0x7f79201278c0 2024-09-15T21:55:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.114+0000 7fdeb1ffb640 1 -- 172.21.15.47:0/2049491447 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fdeac017cd0 con 0x7fdeb408e2e0 2024-09-15T21:55:24.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.114+0000 7fdebc22b640 1 -- 172.21.15.47:0/2049491447 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb404bf60 con 0x7fdeb408e2e0 2024-09-15T21:55:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.115+0000 7f79167fc640 1 -- 172.21.15.47:0/1098625444 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f7918083ff0 con 0x7f79201278c0 2024-09-15T21:55:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.115+0000 7fdeb1ffb640 1 -- 172.21.15.47:0/2049491447 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fdeac068200 con 0x7fdeb408e2e0 2024-09-15T21:55:24.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.118+0000 7f79167fc640 1 -- 172.21.15.47:0/1098625444 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f7918062ee0 con 0x7f79201278c0 2024-09-15T21:55:24.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.118+0000 7fdeb1ffb640 1 -- 172.21.15.47:0/2049491447 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fdeac0430f0 con 0x7fdeb408e2e0 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.119+0000 7ff1e6034640 1 Processor -- start 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.119+0000 7ff1e6034640 1 -- start start 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1e6034640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00979f0 0x7ff1e012a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1e6034640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1e012f180 0x7ff1e0131560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1e6034640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e00990f0 con 0x7ff1e0129c70 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1e6034640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e00998b0 con 0x7ff1e012f180 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1dffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1e012f180 0x7ff1e0131560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1e6034640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e009a180 con 0x7ff1e00979f0 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1dffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1e012f180 0x7ff1e0131560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47994/0 (socket says 172.21.15.47:47994) 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1dffff640 1 -- 172.21.15.47:0/3083803590 learned_addr learned my addr 172.21.15.47:0/3083803590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1deffd640 1 --2- 172.21.15.47:0/3083803590 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00979f0 0x7ff1e012a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1dffff640 1 -- 172.21.15.47:0/3083803590 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00979f0 msgr2=0x7ff1e012a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1dffff640 1 --2- 172.21.15.47:0/3083803590 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00979f0 0x7ff1e012a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1dffff640 1 -- 172.21.15.47:0/3083803590 >> v1:172.21.15.47:6789/0 conn(0x7ff1e0129c70 legacy=0x7ff1e012a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.120+0000 7ff1dffff640 1 -- 172.21.15.47:0/3083803590 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e009a440 con 0x7ff1e012f180 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.121+0000 7ff1dffff640 1 --2- 172.21.15.47:0/3083803590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e012f180 0x7ff1e0131560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009f20 tx=0x7ff1d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500eaeb1a7a4ac86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.121+0000 7ff1de7fc640 1 -- 172.21.15.47:0/3083803590 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8031400 con 0x7ff1e012f180 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.121+0000 7ff1de7fc640 1 -- 172.21.15.47:0/3083803590 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d802d050 con 0x7ff1e012f180 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.121+0000 7ff1de7fc640 1 -- 172.21.15.47:0/3083803590 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80317d0 con 0x7ff1e012f180 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.122+0000 7ff1e6034640 1 -- 172.21.15.47:0/3083803590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e012f180 msgr2=0x7ff1e0131560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.122+0000 7ff1e6034640 1 --2- 172.21.15.47:0/3083803590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e012f180 0x7ff1e0131560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009f20 tx=0x7ff1d8009840 comp rx=0 tx=0).stop 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.122+0000 7ff1e6034640 1 -- 172.21.15.47:0/3083803590 shutdown_connections 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.122+0000 7ff1e6034640 1 --2- 172.21.15.47:0/3083803590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e012f180 0x7ff1e0131560 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.122+0000 7ff1e6034640 1 --2- 172.21.15.47:0/3083803590 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e00979f0 0x7ff1e012a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.122+0000 7ff1e6034640 1 -- 172.21.15.47:0/3083803590 >> 172.21.15.47:0/3083803590 conn(0x7ff1e009d850 msgr2=0x7ff1e009dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.122+0000 7ff1e6034640 1 -- 172.21.15.47:0/3083803590 shutdown_connections 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.122+0000 7ff1e6034640 1 -- 172.21.15.47:0/3083803590 wait complete. 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.123+0000 7ff1e6034640 1 Processor -- start 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.123+0000 7ff1e6034640 1 -- start start 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1e6034640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00979f0 0x7ff1e01395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1e6034640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff1e0129c70 0x7ff1e0139b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1e6034640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e012f180 0x7ff1e0116a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1e6034640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e011a050 con 0x7ff1e00979f0 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1df7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00979f0 0x7ff1e01395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1e6034640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e0119680 con 0x7ff1e012f180 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1e6034640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e0119df0 con 0x7ff1e0129c70 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1deffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff1e0129c70 0x7ff1e0139b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1df7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00979f0 0x7ff1e01395c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48002/0 (socket says 172.21.15.47:48002) 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1df7fe640 1 -- 172.21.15.47:0/2792792352 learned_addr learned my addr 172.21.15.47:0/2792792352 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1dffff640 1 --2- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e012f180 0x7ff1e0116a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1deffd640 1 -- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e012f180 msgr2=0x7ff1e0116a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1deffd640 1 --2- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e012f180 0x7ff1e0116a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1deffd640 1 -- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00979f0 msgr2=0x7ff1e01395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1deffd640 1 --2- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00979f0 0x7ff1e01395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1deffd640 1 -- 172.21.15.47:0/2792792352 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d8009ae0 con 0x7ff1e0129c70 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.124+0000 7ff1deffd640 1 --2- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff1e0129c70 0x7ff1e0139b00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400e900 tx=0x7ff1d40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.125+0000 7ff1c77fe640 1 -- 172.21.15.47:0/2792792352 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40093c0 con 0x7ff1e0129c70 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.125+0000 7ff1c77fe640 1 -- 172.21.15.47:0/2792792352 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d400c940 con 0x7ff1e0129c70 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.125+0000 7ff1c77fe640 1 -- 172.21.15.47:0/2792792352 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400ccc0 con 0x7ff1e0129c70 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.125+0000 7ff1e6034640 1 -- 172.21.15.47:0/2792792352 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e011a230 con 0x7ff1e0129c70 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.126+0000 7ff1e6034640 1 -- 172.21.15.47:0/2792792352 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e0116fd0 con 0x7ff1e0129c70 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.126+0000 7ff1c77fe640 1 -- 172.21.15.47:0/2792792352 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7ff1d4013680 con 0x7ff1e0129c70 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.127+0000 7ff1c77fe640 1 -- 172.21.15.47:0/2792792352 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7ff1d4066d80 con 0x7ff1e0129c70 2024-09-15T21:55:24.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.127+0000 7ff1e6034640 1 -- 172.21.15.47:0/2792792352 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e004bf60 con 0x7ff1e0129c70 2024-09-15T21:55:24.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.131+0000 7ff1c77fe640 1 -- 172.21.15.47:0/2792792352 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7ff1d403db20 con 0x7ff1e0129c70 2024-09-15T21:55:24.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.261+0000 7f811e75d640 1 -- 172.21.15.47:0/3258416272 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f811806d7f0 con 0x7f811806eec0 2024-09-15T21:55:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.261+0000 7fdebc22b640 1 -- 172.21.15.47:0/2049491447 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdeb4126170 con 0x7fdeb408e2e0 2024-09-15T21:55:24.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.261+0000 7f80fffff640 1 -- 172.21.15.47:0/3258416272 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f811806d7f0 con 0x7f811806eec0 2024-09-15T21:55:24.264 INFO:teuthology.orchestra.run.smithi047.stdout:38654705668 2024-09-15T21:55:24.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.262+0000 7fdeb1ffb640 1 -- 172.21.15.47:0/2049491447 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdeac046ad0 con 0x7fdeb408e2e0 2024-09-15T21:55:24.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.262+0000 7f79286dd640 1 -- 172.21.15.47:0/1098625444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7920088c30 con 0x7f79201278c0 2024-09-15T21:55:24.265 INFO:teuthology.orchestra.run.smithi047.stdout:38654705669 2024-09-15T21:55:24.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.262+0000 7f79167fc640 1 -- 172.21.15.47:0/1098625444 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7918040030 con 0x7f79201278c0 2024-09-15T21:55:24.266 INFO:teuthology.orchestra.run.smithi047.stdout:38654705669 2024-09-15T21:55:24.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7f811e75d640 1 -- 172.21.15.47:0/3258416272 >> v1:172.21.15.47:6800/32267 conn(0x7f810404cb70 legacy=0x7f810404f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7f811e75d640 1 -- 172.21.15.47:0/3258416272 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 msgr2=0x7f8118137200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7f811e75d640 1 --2- 172.21.15.47:0/3258416272 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 0x7f8118137200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f810800e960 tx=0x7f81080090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7f811e75d640 1 -- 172.21.15.47:0/3258416272 shutdown_connections 2024-09-15T21:55:24.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7f811e75d640 1 --2- 172.21.15.47:0/3258416272 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8118128670 0x7f8118137c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7f811e75d640 1 --2- 172.21.15.47:0/3258416272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8118128240 0x7f8118137740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7f811e75d640 1 --2- 172.21.15.47:0/3258416272 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f811806eec0 0x7f8118137200 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7f811e75d640 1 -- 172.21.15.47:0/3258416272 >> 172.21.15.47:0/3258416272 conn(0x7f811809b700 msgr2=0x7f81181267b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7f811e75d640 1 -- 172.21.15.47:0/3258416272 shutdown_connections 2024-09-15T21:55:24.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7f811e75d640 1 -- 172.21.15.47:0/3258416272 wait complete. 2024-09-15T21:55:24.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7fdebc22b640 1 -- 172.21.15.47:0/2049491447 >> v1:172.21.15.47:6800/32267 conn(0x7fdea004cba0 legacy=0x7fdea004f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7fdebc22b640 1 -- 172.21.15.47:0/2049491447 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 msgr2=0x7fdeb4124710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.264+0000 7fdebc22b640 1 --2- 172.21.15.47:0/2049491447 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb4124710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0124f0 tx=0x7fdeac00db90 comp rx=0 tx=0).stop 2024-09-15T21:55:24.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7fdebc22b640 1 -- 172.21.15.47:0/2049491447 shutdown_connections 2024-09-15T21:55:24.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7fdebc22b640 1 --2- 172.21.15.47:0/2049491447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdeb412b710 0x7fdeb4125190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7fdebc22b640 1 --2- 172.21.15.47:0/2049491447 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdeb408ebf0 0x7fdeb4124c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7fdebc22b640 1 --2- 172.21.15.47:0/2049491447 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdeb408e2e0 0x7fdeb4124710 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7fdebc22b640 1 -- 172.21.15.47:0/2049491447 >> 172.21.15.47:0/2049491447 conn(0x7fdeb409ac50 msgr2=0x7fdeb4097f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7fdebc22b640 1 -- 172.21.15.47:0/2049491447 shutdown_connections 2024-09-15T21:55:24.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7fdebc22b640 1 -- 172.21.15.47:0/2049491447 wait complete. 2024-09-15T21:55:24.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7f79286dd640 1 -- 172.21.15.47:0/1098625444 >> v1:172.21.15.47:6800/32267 conn(0x7f790404cbd0 legacy=0x7f790404f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7f79286dd640 1 -- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79201278c0 msgr2=0x7f7920135010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7f79286dd640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79201278c0 0x7f7920135010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f792006da50 tx=0x7f7918036000 comp rx=0 tx=0).stop 2024-09-15T21:55:24.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7f79286dd640 1 -- 172.21.15.47:0/1098625444 shutdown_connections 2024-09-15T21:55:24.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7f79286dd640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79201278c0 0x7f7920135010 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7f79286dd640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7920096160 0x7f7920127380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7f79286dd640 1 --2- 172.21.15.47:0/1098625444 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7920095850 0x7f7920126e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7f79286dd640 1 -- 172.21.15.47:0/1098625444 >> 172.21.15.47:0/1098625444 conn(0x7f7920096610 msgr2=0x7f7920098380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7f79286dd640 1 -- 172.21.15.47:0/1098625444 shutdown_connections 2024-09-15T21:55:24.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.265+0000 7f79286dd640 1 -- 172.21.15.47:0/1098625444 wait complete. 2024-09-15T21:55:24.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.3 2024-09-15T21:55:24.281 DEBUG:teuthology.parallel:result is None 2024-09-15T21:55:24.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.0 2024-09-15T21:55:24.281 DEBUG:teuthology.parallel:result is None 2024-09-15T21:55:24.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.2 2024-09-15T21:55:24.282 DEBUG:teuthology.parallel:result is None 2024-09-15T21:55:24.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.300+0000 7ff1e6034640 1 -- 172.21.15.47:0/2792792352 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff1e00646b0 con 0x7ff1e0129c70 2024-09-15T21:55:24.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.300+0000 7ff1c77fe640 1 -- 172.21.15.47:0/2792792352 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff1d4041330 con 0x7ff1e0129c70 2024-09-15T21:55:24.303 INFO:teuthology.orchestra.run.smithi047.stdout:38654705668 2024-09-15T21:55:24.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.302+0000 7ff1e6034640 1 -- 172.21.15.47:0/2792792352 >> v1:172.21.15.47:6800/32267 conn(0x7ff1cc04cb70 legacy=0x7ff1cc04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.302+0000 7ff1e6034640 1 -- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff1e0129c70 msgr2=0x7ff1e0139b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.302+0000 7ff1e6034640 1 --2- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff1e0129c70 0x7ff1e0139b00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400e900 tx=0x7ff1d40090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.302+0000 7ff1e6034640 1 -- 172.21.15.47:0/2792792352 shutdown_connections 2024-09-15T21:55:24.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.302+0000 7ff1e6034640 1 --2- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff1e012f180 0x7ff1e0116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.302+0000 7ff1e6034640 1 --2- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff1e0129c70 0x7ff1e0139b00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.302+0000 7ff1e6034640 1 --2- 172.21.15.47:0/2792792352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1e00979f0 0x7ff1e01395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.302+0000 7ff1e6034640 1 -- 172.21.15.47:0/2792792352 >> 172.21.15.47:0/2792792352 conn(0x7ff1e009d850 msgr2=0x7ff1e012ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.302+0000 7ff1e6034640 1 -- 172.21.15.47:0/2792792352 shutdown_connections 2024-09-15T21:55:24.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.302+0000 7ff1e6034640 1 -- 172.21.15.47:0/2792792352 wait complete. 2024-09-15T21:55:24.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.1 2024-09-15T21:55:24.316 DEBUG:teuthology.parallel:result is None 2024-09-15T21:55:24.316 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:55:24.316 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:55:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.450+0000 7f963f846640 1 Processor -- start 2024-09-15T21:55:24.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.450+0000 7f963f846640 1 -- start start 2024-09-15T21:55:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963f846640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 0x7f963812a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963f846640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f96380970f0 0x7f9638097590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963f846640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96380998b0 con 0x7f963812a580 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963f846640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963809a180 con 0x7f96380970f0 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963f846640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f963809a440 con 0x7f9638129c70 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963d5e2640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 0x7f963812a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963cde1640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f963812a580 0x7f96380969e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53082/0 (socket says 172.21.15.47:53082) 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963dde3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f96380970f0 0x7f9638097590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963cde1640 1 -- 172.21.15.47:0/931107236 learned_addr learned my addr 172.21.15.47:0/931107236 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f962ffff640 1 -- 172.21.15.47:0/931107236 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312636850 0 0) 0x7f96380998b0 con 0x7f963812a580 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f962ffff640 1 -- 172.21.15.47:0/931107236 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96240034e0 con 0x7f963812a580 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963d5e2640 1 -- 172.21.15.47:0/931107236 >> v2:172.21.15.47:3300/0 conn(0x7f96380970f0 msgr2=0x7f9638097590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963d5e2640 1 --2- 172.21.15.47:0/931107236 >> v2:172.21.15.47:3300/0 conn(0x7f96380970f0 0x7f9638097590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963d5e2640 1 -- 172.21.15.47:0/931107236 >> v1:172.21.15.47:6789/0 conn(0x7f963812a580 legacy=0x7f96380969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963d5e2640 1 -- 172.21.15.47:0/931107236 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9638099db0 con 0x7f9638129c70 2024-09-15T21:55:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.451+0000 7f963dde3640 1 --2- 172.21.15.47:0/931107236 >> v2:172.21.15.47:3300/0 conn(0x7f96380970f0 0x7f9638097590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.452+0000 7f963d5e2640 1 --2- 172.21.15.47:0/931107236 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 0x7f963812a040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9634009850 tx=0x7f96340090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28478425bd4c0ff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.452+0000 7f962ffff640 1 -- 172.21.15.47:0/931107236 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963402ac80 con 0x7f9638129c70 2024-09-15T21:55:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.452+0000 7f962ffff640 1 -- 172.21.15.47:0/931107236 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963402ade0 con 0x7f9638129c70 2024-09-15T21:55:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.452+0000 7f962ffff640 1 -- 172.21.15.47:0/931107236 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634032650 con 0x7f9638129c70 2024-09-15T21:55:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.453+0000 7f963f846640 1 -- 172.21.15.47:0/931107236 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 msgr2=0x7f963812a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.453+0000 7f963f846640 1 --2- 172.21.15.47:0/931107236 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 0x7f963812a040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9634009850 tx=0x7f96340090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.453+0000 7f963f846640 1 -- 172.21.15.47:0/931107236 shutdown_connections 2024-09-15T21:55:24.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.453+0000 7f963f846640 1 --2- 172.21.15.47:0/931107236 >> v2:172.21.15.47:3300/0 conn(0x7f96380970f0 0x7f9638097590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.453+0000 7f963f846640 1 --2- 172.21.15.47:0/931107236 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 0x7f963812a040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.453+0000 7f963f846640 1 -- 172.21.15.47:0/931107236 >> 172.21.15.47:0/931107236 conn(0x7f963809d850 msgr2=0x7f963809dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.453+0000 7f963f846640 1 -- 172.21.15.47:0/931107236 shutdown_connections 2024-09-15T21:55:24.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.453+0000 7f963f846640 1 -- 172.21.15.47:0/931107236 wait complete. 2024-09-15T21:55:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963f846640 1 Processor -- start 2024-09-15T21:55:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963f846640 1 -- start start 2024-09-15T21:55:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963f846640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96380970f0 0x7f9638139470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963f846640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 0x7f96381399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963f846640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963812a580 0x7f963811a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963f846640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963811c7c0 con 0x7f963812a580 2024-09-15T21:55:24.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963f846640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963811dcb0 con 0x7f96380970f0 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963f846640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f963811dee0 con 0x7f9638129c70 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963d5e2640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96380970f0 0x7f9638139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963cde1640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 0x7f96381399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963dde3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963812a580 0x7f963811a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963d5e2640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96380970f0 0x7f9638139470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52562/0 (socket says 172.21.15.47:52562) 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963d5e2640 1 -- 172.21.15.47:0/500444558 learned_addr learned my addr 172.21.15.47:0/500444558 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.454+0000 7f963cde1640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 0x7f96381399b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:48288/0 (socket says 172.21.15.47:48288) 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f963d5e2640 1 -- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 msgr2=0x7f96381399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f963d5e2640 1 --2- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 0x7f96381399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f963d5e2640 1 -- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963812a580 msgr2=0x7f963811a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f963d5e2640 1 --2- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963812a580 0x7f963811a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f963d5e2640 1 -- 172.21.15.47:0/500444558 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630009a20 con 0x7f96380970f0 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f963dde3640 1 --2- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963812a580 0x7f963811a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f963d5e2640 1 --2- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96380970f0 0x7f9638139470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9634009130 tx=0x7f9634032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f962d7fa640 1 -- 172.21.15.47:0/500444558 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963403c720 con 0x7f96380970f0 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f962d7fa640 1 -- 172.21.15.47:0/500444558 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963403c880 con 0x7f96380970f0 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f963f846640 1 -- 172.21.15.47:0/500444558 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9634009390 con 0x7f96380970f0 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f962d7fa640 1 -- 172.21.15.47:0/500444558 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963403cc40 con 0x7f96380970f0 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.455+0000 7f963f846640 1 -- 172.21.15.47:0/500444558 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963811d3c0 con 0x7f96380970f0 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.456+0000 7f963f846640 1 -- 172.21.15.47:0/500444558 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963811d610 con 0x7f96380970f0 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.456+0000 7f962d7fa640 1 -- 172.21.15.47:0/500444558 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f96340315b0 con 0x7f96380970f0 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.457+0000 7f962d7fa640 1 -- 172.21.15.47:0/500444558 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f9634083820 con 0x7f96380970f0 2024-09-15T21:55:24.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.460+0000 7f962d7fa640 1 -- 172.21.15.47:0/500444558 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f9634062710 con 0x7f96380970f0 2024-09-15T21:55:24.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.595+0000 7f963f846640 1 -- 172.21.15.47:0/500444558 --> v1:172.21.15.47:6800/32267 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f963804bf60 con 0x7f961c04cb70 2024-09-15T21:55:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.596+0000 7f962d7fa640 1 -- 172.21.15.47:0/500444558 <== mgr.4100 v1:172.21.15.47:6800/32267 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34999 (unknown 2965378022 0 56742415) 0x7f963804bf60 con 0x7f961c04cb70 2024-09-15T21:55:24.599 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:55:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:55:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.598+0000 7f963f846640 1 -- 172.21.15.47:0/500444558 >> v1:172.21.15.47:6800/32267 conn(0x7f961c04cb70 legacy=0x7f961c04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.598+0000 7f963f846640 1 -- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96380970f0 msgr2=0x7f9638139470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.598+0000 7f963f846640 1 --2- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96380970f0 0x7f9638139470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9634009130 tx=0x7f9634032000 comp rx=0 tx=0).stop 2024-09-15T21:55:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.598+0000 7f963f846640 1 -- 172.21.15.47:0/500444558 shutdown_connections 2024-09-15T21:55:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.598+0000 7f963f846640 1 --2- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963812a580 0x7f963811a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.598+0000 7f963f846640 1 --2- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9638129c70 0x7f96381399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.598+0000 7f963f846640 1 --2- 172.21.15.47:0/500444558 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f96380970f0 0x7f9638139470 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.598+0000 7f963f846640 1 -- 172.21.15.47:0/500444558 >> 172.21.15.47:0/500444558 conn(0x7f963809d850 msgr2=0x7f963812cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.598+0000 7f963f846640 1 -- 172.21.15.47:0/500444558 shutdown_connections 2024-09-15T21:55:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.598+0000 7f963f846640 1 -- 172.21.15.47:0/500444558 wait complete. 2024-09-15T21:55:24.610 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-15T21:55:22.696590+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1207780,"kb_used_data":1696,"kb_used_omap":6,"kb_used_meta":1206009,"kb_avail":376279580,"statfs":{"total":386547056640,"available":385310289920,"internally_reserved":0,"allocated":1736704,"data_stored":1329010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1234953568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.429076"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.285832+0000","last_change":"2024-09-15T21:55:20.285996+0000","last_active":"2024-09-15T21:55:20.285832+0000","last_peered":"2024-09-15T21:55:20.285832+0000","last_clean":"2024-09-15T21:55:20.285832+0000","last_became_active":"2024-09-15T21:55:18.278669+0000","last_became_peered":"2024-09-15T21:55:18.278669+0000","last_unstale":"2024-09-15T21:55:20.285832+0000","last_undegraded":"2024-09-15T21:55:20.285832+0000","last_fullsized":"2024-09-15T21:55:20.285832+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:25.111524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.280473+0000","last_change":"2024-09-15T21:55:20.281027+0000","last_active":"2024-09-15T21:55:20.280473+0000","last_peered":"2024-09-15T21:55:20.280473+0000","last_clean":"2024-09-15T21:55:20.280473+0000","last_became_active":"2024-09-15T21:55:18.280627+0000","last_became_peered":"2024-09-15T21:55:18.280627+0000","last_unstale":"2024-09-15T21:55:20.280473+0000","last_undegraded":"2024-09-15T21:55:20.280473+0000","last_fullsized":"2024-09-15T21:55:20.280473+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:22.442431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.286372+0000","last_change":"2024-09-15T21:55:20.286607+0000","last_active":"2024-09-15T21:55:20.286372+0000","last_peered":"2024-09-15T21:55:20.286372+0000","last_clean":"2024-09-15T21:55:20.286372+0000","last_became_active":"2024-09-15T21:55:18.277825+0000","last_became_peered":"2024-09-15T21:55:18.277825+0000","last_unstale":"2024-09-15T21:55:20.286372+0000","last_undegraded":"2024-09-15T21:55:20.286372+0000","last_fullsized":"2024-09-15T21:55:20.286372+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:22.442431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.280283+0000","last_change":"2024-09-15T21:55:20.280811+0000","last_active":"2024-09-15T21:55:20.280283+0000","last_peered":"2024-09-15T21:55:20.280283+0000","last_clean":"2024-09-15T21:55:20.280283+0000","last_became_active":"2024-09-15T21:55:18.279130+0000","last_became_peered":"2024-09-15T21:55:18.279130+0000","last_unstale":"2024-09-15T21:55:20.280283+0000","last_undegraded":"2024-09-15T21:55:20.280283+0000","last_fullsized":"2024-09-15T21:55:20.280283+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:25.111524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.286526+0000","last_change":"2024-09-15T21:55:20.286526+0000","last_active":"2024-09-15T21:55:20.286526+0000","last_peered":"2024-09-15T21:55:20.286526+0000","last_clean":"2024-09-15T21:55:20.286526+0000","last_became_active":"2024-09-15T21:55:18.279213+0000","last_became_peered":"2024-09-15T21:55:18.279213+0000","last_unstale":"2024-09-15T21:55:20.286526+0000","last_undegraded":"2024-09-15T21:55:20.286526+0000","last_fullsized":"2024-09-15T21:55:20.286526+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:01.390413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.280568+0000","last_change":"2024-09-15T21:55:20.280869+0000","last_active":"2024-09-15T21:55:20.280568+0000","last_peered":"2024-09-15T21:55:20.280568+0000","last_clean":"2024-09-15T21:55:20.280568+0000","last_became_active":"2024-09-15T21:55:18.276731+0000","last_became_peered":"2024-09-15T21:55:18.276731+0000","last_unstale":"2024-09-15T21:55:20.280568+0000","last_undegraded":"2024-09-15T21:55:20.280568+0000","last_fullsized":"2024-09-15T21:55:20.280568+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:04.981724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.285774+0000","last_change":"2024-09-15T21:55:20.285977+0000","last_active":"2024-09-15T21:55:20.285774+0000","last_peered":"2024-09-15T21:55:20.285774+0000","last_clean":"2024-09-15T21:55:20.285774+0000","last_became_active":"2024-09-15T21:55:18.280367+0000","last_became_peered":"2024-09-15T21:55:18.280367+0000","last_unstale":"2024-09-15T21:55:20.285774+0000","last_undegraded":"2024-09-15T21:55:20.285774+0000","last_fullsized":"2024-09-15T21:55:20.285774+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:03.529028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.286583+0000","last_change":"2024-09-15T21:55:20.286832+0000","last_active":"2024-09-15T21:55:20.286583+0000","last_peered":"2024-09-15T21:55:20.286583+0000","last_clean":"2024-09-15T21:55:20.286583+0000","last_became_active":"2024-09-15T21:55:18.278579+0000","last_became_peered":"2024-09-15T21:55:18.278579+0000","last_unstale":"2024-09-15T21:55:20.286583+0000","last_undegraded":"2024-09-15T21:55:20.286583+0000","last_fullsized":"2024-09-15T21:55:20.286583+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:15:44.779410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"11'52","reported_seq":80,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:55:19.283560+0000","last_change":"2024-09-15T21:55:12.324328+0000","last_active":"2024-09-15T21:55:19.283560+0000","last_peered":"2024-09-15T21:55:19.283560+0000","last_clean":"2024-09-15T21:55:19.283560+0000","last_became_active":"2024-09-15T21:55:12.323202+0000","last_became_peered":"2024-09-15T21:55:12.323202+0000","last_unstale":"2024-09-15T21:55:19.283560+0000","last_undegraded":"2024-09-15T21:55:19.283560+0000","last_fullsized":"2024-09-15T21:55:19.283560+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_clean_scrub_stamp":"2024-09-15T21:55:11.249936+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:04.912041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301664,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070176,"statfs":{"total":96636764160,"available":96327860224,"internally_reserved":0,"allocated":143360,"data_stored":38789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301648,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070192,"statfs":{"total":96636764160,"available":96327876608,"internally_reserved":0,"allocated":131072,"data_stored":35348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302236,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069604,"statfs":{"total":96636764160,"available":96327274496,"internally_reserved":0,"allocated":737280,"data_stored":629157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302232,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069608,"statfs":{"total":96636764160,"available":96327278592,"internally_reserved":0,"allocated":724992,"data_stored":625716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:55:24.611 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:55:24.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.705+0000 7fe4ab280640 1 Processor -- start 2024-09-15T21:55:24.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.705+0000 7fe4ab280640 1 -- start start 2024-09-15T21:55:24.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4ab280640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4129c70 0x7fe4a412a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4ab280640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 0x7fe4a40969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4ab280640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a40998b0 con 0x7fe4a4096f20 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4ab280640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a409a180 con 0x7fe4a4129c70 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4ab280640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a409a440 con 0x7fe4a412a580 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4a881b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 0x7fe4a40969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4a901c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4129c70 0x7fe4a412a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4a981d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe4a4096f20 0x7fe4a4097370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53088/0 (socket says 172.21.15.47:53088) 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4a981d640 1 -- 172.21.15.47:0/161532085 learned_addr learned my addr 172.21.15.47:0/161532085 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe49bfff640 1 -- 172.21.15.47:0/161532085 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886751357 0 0) 0x7fe4a40998b0 con 0x7fe4a4096f20 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4a881b640 1 -- 172.21.15.47:0/161532085 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4129c70 msgr2=0x7fe4a412a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4a881b640 1 --2- 172.21.15.47:0/161532085 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4129c70 0x7fe4a412a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4a881b640 1 -- 172.21.15.47:0/161532085 >> v1:172.21.15.47:6789/0 conn(0x7fe4a4096f20 legacy=0x7fe4a4097370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4a881b640 1 -- 172.21.15.47:0/161532085 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a4099db0 con 0x7fe4a412a580 2024-09-15T21:55:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.706+0000 7fe4a901c640 1 --2- 172.21.15.47:0/161532085 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4129c70 0x7fe4a412a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:24.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.707+0000 7fe4a881b640 1 --2- 172.21.15.47:0/161532085 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 0x7fe4a40969e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe494009880 tx=0x7fe4940090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2f5603feb31a952 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.707+0000 7fe49bfff640 1 -- 172.21.15.47:0/161532085 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49402ac50 con 0x7fe4a412a580 2024-09-15T21:55:24.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.707+0000 7fe49bfff640 1 -- 172.21.15.47:0/161532085 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49402adb0 con 0x7fe4a412a580 2024-09-15T21:55:24.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.707+0000 7fe49bfff640 1 -- 172.21.15.47:0/161532085 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494032670 con 0x7fe4a412a580 2024-09-15T21:55:24.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.708+0000 7fe4ab280640 1 -- 172.21.15.47:0/161532085 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 msgr2=0x7fe4a40969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.708+0000 7fe4ab280640 1 --2- 172.21.15.47:0/161532085 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 0x7fe4a40969e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe494009880 tx=0x7fe4940090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.708+0000 7fe4ab280640 1 -- 172.21.15.47:0/161532085 shutdown_connections 2024-09-15T21:55:24.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.708+0000 7fe4ab280640 1 --2- 172.21.15.47:0/161532085 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 0x7fe4a40969e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.708+0000 7fe4ab280640 1 --2- 172.21.15.47:0/161532085 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4129c70 0x7fe4a412a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.708+0000 7fe4ab280640 1 -- 172.21.15.47:0/161532085 >> 172.21.15.47:0/161532085 conn(0x7fe4a409d850 msgr2=0x7fe4a409dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.708+0000 7fe4ab280640 1 -- 172.21.15.47:0/161532085 shutdown_connections 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.708+0000 7fe4ab280640 1 -- 172.21.15.47:0/161532085 wait complete. 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.708+0000 7fe4ab280640 1 Processor -- start 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4ab280640 1 -- start start 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4ab280640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4096f20 0x7fe4a4139450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4ab280640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4129c70 0x7fe4a4139990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4ab280640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 0x7fe4a411a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4ab280640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a412cd40 con 0x7fe4a4129c70 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4ab280640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a411d6d0 con 0x7fe4a4096f20 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4ab280640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a411dc70 con 0x7fe4a412a580 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4a901c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4096f20 0x7fe4a4139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4a901c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4096f20 0x7fe4a4139450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52586/0 (socket says 172.21.15.47:52586) 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4a901c640 1 -- 172.21.15.47:0/3334582514 learned_addr learned my addr 172.21.15.47:0/3334582514 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4a981d640 1 --2- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 0x7fe4a411a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.709+0000 7fe4a881b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4129c70 0x7fe4a4139990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4a901c640 1 -- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 msgr2=0x7fe4a411a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4a901c640 1 --2- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 0x7fe4a411a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4a901c640 1 -- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4129c70 msgr2=0x7fe4a4139990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4a901c640 1 --2- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4129c70 0x7fe4a4139990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4a901c640 1 -- 172.21.15.47:0/3334582514 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a0009a20 con 0x7fe4a4096f20 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4a981d640 1 --2- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 0x7fe4a411a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4a901c640 1 --2- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4096f20 0x7fe4a4139450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000c170 tx=0x7fe4a000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4997fa640 1 -- 172.21.15.47:0/3334582514 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0014480 con 0x7fe4a4096f20 2024-09-15T21:55:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4997fa640 1 -- 172.21.15.47:0/3334582514 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a000f040 con 0x7fe4a4096f20 2024-09-15T21:55:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4997fa640 1 -- 172.21.15.47:0/3334582514 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0014810 con 0x7fe4a4096f20 2024-09-15T21:55:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4ab280640 1 -- 172.21.15.47:0/3334582514 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe494009390 con 0x7fe4a4096f20 2024-09-15T21:55:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.710+0000 7fe4ab280640 1 -- 172.21.15.47:0/3334582514 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a4139ed0 con 0x7fe4a4096f20 2024-09-15T21:55:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.711+0000 7fe4997fa640 1 -- 172.21.15.47:0/3334582514 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fe4a0014b80 con 0x7fe4a4096f20 2024-09-15T21:55:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.711+0000 7fe4ab280640 1 -- 172.21.15.47:0/3334582514 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a404bf60 con 0x7fe4a4096f20 2024-09-15T21:55:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.711+0000 7fe4997fa640 1 -- 172.21.15.47:0/3334582514 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fe4a00640d0 con 0x7fe4a4096f20 2024-09-15T21:55:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.715+0000 7fe4997fa640 1 -- 172.21.15.47:0/3334582514 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fe4a003f210 con 0x7fe4a4096f20 2024-09-15T21:55:24.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.851+0000 7fe4ab280640 1 -- 172.21.15.47:0/3334582514 --> v1:172.21.15.47:6800/32267 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4a40646b0 con 0x7fe48804cb70 2024-09-15T21:55:24.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.853+0000 7fe4997fa640 1 -- 172.21.15.47:0/3334582514 <== mgr.4100 v1:172.21.15.47:6800/32267 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34999 (unknown 2965378022 0 4286931286) 0x7fe4a40646b0 con 0x7fe48804cb70 2024-09-15T21:55:24.856 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:55:24.856 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:55:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.854+0000 7fe4ab280640 1 -- 172.21.15.47:0/3334582514 >> v1:172.21.15.47:6800/32267 conn(0x7fe48804cb70 legacy=0x7fe48804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.854+0000 7fe4ab280640 1 -- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4096f20 msgr2=0x7fe4a4139450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.854+0000 7fe4ab280640 1 --2- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4096f20 0x7fe4a4139450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000c170 tx=0x7fe4a000b040 comp rx=0 tx=0).stop 2024-09-15T21:55:24.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.854+0000 7fe4ab280640 1 -- 172.21.15.47:0/3334582514 shutdown_connections 2024-09-15T21:55:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.854+0000 7fe4ab280640 1 --2- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe4a412a580 0x7fe4a411a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.854+0000 7fe4ab280640 1 --2- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4129c70 0x7fe4a4139990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.854+0000 7fe4ab280640 1 --2- 172.21.15.47:0/3334582514 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe4a4096f20 0x7fe4a4139450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.854+0000 7fe4ab280640 1 -- 172.21.15.47:0/3334582514 >> 172.21.15.47:0/3334582514 conn(0x7fe4a409d850 msgr2=0x7fe4a412cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.854+0000 7fe4ab280640 1 -- 172.21.15.47:0/3334582514 shutdown_connections 2024-09-15T21:55:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.854+0000 7fe4ab280640 1 -- 172.21.15.47:0/3334582514 wait complete. 2024-09-15T21:55:24.867 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-15T21:55:24.696974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1207780,"kb_used_data":1696,"kb_used_omap":6,"kb_used_meta":1206009,"kb_avail":376279580,"statfs":{"total":386547056640,"available":385310289920,"internally_reserved":0,"allocated":1736704,"data_stored":1329010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1234953568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.427967"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.285832+0000","last_change":"2024-09-15T21:55:20.285996+0000","last_active":"2024-09-15T21:55:20.285832+0000","last_peered":"2024-09-15T21:55:20.285832+0000","last_clean":"2024-09-15T21:55:20.285832+0000","last_became_active":"2024-09-15T21:55:18.278669+0000","last_became_peered":"2024-09-15T21:55:18.278669+0000","last_unstale":"2024-09-15T21:55:20.285832+0000","last_undegraded":"2024-09-15T21:55:20.285832+0000","last_fullsized":"2024-09-15T21:55:20.285832+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:25.111524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.280473+0000","last_change":"2024-09-15T21:55:20.281027+0000","last_active":"2024-09-15T21:55:20.280473+0000","last_peered":"2024-09-15T21:55:20.280473+0000","last_clean":"2024-09-15T21:55:20.280473+0000","last_became_active":"2024-09-15T21:55:18.280627+0000","last_became_peered":"2024-09-15T21:55:18.280627+0000","last_unstale":"2024-09-15T21:55:20.280473+0000","last_undegraded":"2024-09-15T21:55:20.280473+0000","last_fullsized":"2024-09-15T21:55:20.280473+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:22.442431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.286372+0000","last_change":"2024-09-15T21:55:20.286607+0000","last_active":"2024-09-15T21:55:20.286372+0000","last_peered":"2024-09-15T21:55:20.286372+0000","last_clean":"2024-09-15T21:55:20.286372+0000","last_became_active":"2024-09-15T21:55:18.277825+0000","last_became_peered":"2024-09-15T21:55:18.277825+0000","last_unstale":"2024-09-15T21:55:20.286372+0000","last_undegraded":"2024-09-15T21:55:20.286372+0000","last_fullsized":"2024-09-15T21:55:20.286372+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:22.442431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.280283+0000","last_change":"2024-09-15T21:55:20.280811+0000","last_active":"2024-09-15T21:55:20.280283+0000","last_peered":"2024-09-15T21:55:20.280283+0000","last_clean":"2024-09-15T21:55:20.280283+0000","last_became_active":"2024-09-15T21:55:18.279130+0000","last_became_peered":"2024-09-15T21:55:18.279130+0000","last_unstale":"2024-09-15T21:55:20.280283+0000","last_undegraded":"2024-09-15T21:55:20.280283+0000","last_fullsized":"2024-09-15T21:55:20.280283+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:25.111524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.286526+0000","last_change":"2024-09-15T21:55:20.286526+0000","last_active":"2024-09-15T21:55:20.286526+0000","last_peered":"2024-09-15T21:55:20.286526+0000","last_clean":"2024-09-15T21:55:20.286526+0000","last_became_active":"2024-09-15T21:55:18.279213+0000","last_became_peered":"2024-09-15T21:55:18.279213+0000","last_unstale":"2024-09-15T21:55:20.286526+0000","last_undegraded":"2024-09-15T21:55:20.286526+0000","last_fullsized":"2024-09-15T21:55:20.286526+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:01.390413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.280568+0000","last_change":"2024-09-15T21:55:20.280869+0000","last_active":"2024-09-15T21:55:20.280568+0000","last_peered":"2024-09-15T21:55:20.280568+0000","last_clean":"2024-09-15T21:55:20.280568+0000","last_became_active":"2024-09-15T21:55:18.276731+0000","last_became_peered":"2024-09-15T21:55:18.276731+0000","last_unstale":"2024-09-15T21:55:20.280568+0000","last_undegraded":"2024-09-15T21:55:20.280568+0000","last_fullsized":"2024-09-15T21:55:20.280568+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:04.981724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.285774+0000","last_change":"2024-09-15T21:55:20.285977+0000","last_active":"2024-09-15T21:55:20.285774+0000","last_peered":"2024-09-15T21:55:20.285774+0000","last_clean":"2024-09-15T21:55:20.285774+0000","last_became_active":"2024-09-15T21:55:18.280367+0000","last_became_peered":"2024-09-15T21:55:18.280367+0000","last_unstale":"2024-09-15T21:55:20.285774+0000","last_undegraded":"2024-09-15T21:55:20.285774+0000","last_fullsized":"2024-09-15T21:55:20.285774+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:03.529028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T21:55:20.286583+0000","last_change":"2024-09-15T21:55:20.286832+0000","last_active":"2024-09-15T21:55:20.286583+0000","last_peered":"2024-09-15T21:55:20.286583+0000","last_clean":"2024-09-15T21:55:20.286583+0000","last_became_active":"2024-09-15T21:55:18.278579+0000","last_became_peered":"2024-09-15T21:55:18.278579+0000","last_unstale":"2024-09-15T21:55:20.286583+0000","last_undegraded":"2024-09-15T21:55:20.286583+0000","last_fullsized":"2024-09-15T21:55:20.286583+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:15:44.779410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"11'52","reported_seq":80,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T21:55:19.283560+0000","last_change":"2024-09-15T21:55:12.324328+0000","last_active":"2024-09-15T21:55:19.283560+0000","last_peered":"2024-09-15T21:55:19.283560+0000","last_clean":"2024-09-15T21:55:19.283560+0000","last_became_active":"2024-09-15T21:55:12.323202+0000","last_became_peered":"2024-09-15T21:55:12.323202+0000","last_unstale":"2024-09-15T21:55:19.283560+0000","last_undegraded":"2024-09-15T21:55:19.283560+0000","last_fullsized":"2024-09-15T21:55:19.283560+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_clean_scrub_stamp":"2024-09-15T21:55:11.249936+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:04.912041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301664,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070176,"statfs":{"total":96636764160,"available":96327860224,"internally_reserved":0,"allocated":143360,"data_stored":38789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301648,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070192,"statfs":{"total":96636764160,"available":96327876608,"internally_reserved":0,"allocated":131072,"data_stored":35348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302236,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069604,"statfs":{"total":96636764160,"available":96327274496,"internally_reserved":0,"allocated":737280,"data_stored":629157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302232,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069608,"statfs":{"total":96636764160,"available":96327278592,"internally_reserved":0,"allocated":724992,"data_stored":625716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:55:24.868 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:55:24.868 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T21:55:24.868 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T21:55:24.868 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:55:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.961+0000 7f47da06f640 1 Processor -- start 2024-09-15T21:55:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.961+0000 7f47da06f640 1 -- start start 2024-09-15T21:55:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.961+0000 7f47da06f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4096540 0x7f47d412d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.961+0000 7f47da06f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d412d730 0x7f47d412fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.961+0000 7f47da06f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d4093a60 con 0x7f47d4095a60 2024-09-15T21:55:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.961+0000 7f47da06f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d4096930 con 0x7f47d412d730 2024-09-15T21:55:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.961+0000 7f47da06f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4090d00 con 0x7f47d4096540 2024-09-15T21:55:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.961+0000 7f47d3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d412d730 0x7f47d412fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.961+0000 7f47d2ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4096540 0x7f47d412d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.962+0000 7f47d2ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4096540 0x7f47d412d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52590/0 (socket says 172.21.15.47:52590) 2024-09-15T21:55:24.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.962+0000 7f47d3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47d412d730 0x7f47d412fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48028/0 (socket says 172.21.15.47:48028) 2024-09-15T21:55:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.962+0000 7f47d2ffd640 1 -- 172.21.15.47:0/3251041527 learned_addr learned my addr 172.21.15.47:0/3251041527 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.962+0000 7f47d37fe640 1 --1- 172.21.15.47:0/3251041527 >> v1:172.21.15.47:6789/0 conn(0x7f47d4095a60 0x7f47d4095e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53092/0 (socket says 172.21.15.47:53092) 2024-09-15T21:55:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.962+0000 7f47d2ffd640 1 -- 172.21.15.47:0/3251041527 >> v2:172.21.15.47:3300/0 conn(0x7f47d412d730 msgr2=0x7f47d412fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.962+0000 7f47d2ffd640 1 --2- 172.21.15.47:0/3251041527 >> v2:172.21.15.47:3300/0 conn(0x7f47d412d730 0x7f47d412fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.962+0000 7f47d2ffd640 1 -- 172.21.15.47:0/3251041527 >> v1:172.21.15.47:6789/0 conn(0x7f47d4095a60 legacy=0x7f47d4095e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.962+0000 7f47d27fc640 1 -- 172.21.15.47:0/3251041527 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614618099 0 0) 0x7f47d4093a60 con 0x7f47d4095a60 2024-09-15T21:55:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.962+0000 7f47d2ffd640 1 -- 172.21.15.47:0/3251041527 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4094530 con 0x7f47d4096540 2024-09-15T21:55:24.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.962+0000 7f47d3fff640 1 --2- 172.21.15.47:0/3251041527 >> v2:172.21.15.47:3300/0 conn(0x7f47d412d730 0x7f47d412fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47d2ffd640 1 --2- 172.21.15.47:0/3251041527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4096540 0x7f47d412d1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009910 tx=0x7f47c80091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e72ebb21bd47fb74 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47d27fc640 1 -- 172.21.15.47:0/3251041527 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c802ad20 con 0x7f47d4096540 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47d27fc640 1 -- 172.21.15.47:0/3251041527 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c802ae80 con 0x7f47d4096540 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47d27fc640 1 -- 172.21.15.47:0/3251041527 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8032670 con 0x7f47d4096540 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47da06f640 1 -- 172.21.15.47:0/3251041527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4096540 msgr2=0x7f47d412d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47da06f640 1 --2- 172.21.15.47:0/3251041527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4096540 0x7f47d412d1f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009910 tx=0x7f47c80091d0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47da06f640 1 -- 172.21.15.47:0/3251041527 shutdown_connections 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47da06f640 1 --2- 172.21.15.47:0/3251041527 >> v2:172.21.15.47:3300/0 conn(0x7f47d412d730 0x7f47d412fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47da06f640 1 --2- 172.21.15.47:0/3251041527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4096540 0x7f47d412d1f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47da06f640 1 -- 172.21.15.47:0/3251041527 >> 172.21.15.47:0/3251041527 conn(0x7f47d409ac50 msgr2=0x7f47d4096b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47da06f640 1 -- 172.21.15.47:0/3251041527 shutdown_connections 2024-09-15T21:55:24.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.963+0000 7f47da06f640 1 -- 172.21.15.47:0/3251041527 wait complete. 2024-09-15T21:55:24.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.964+0000 7f47da06f640 1 Processor -- start 2024-09-15T21:55:24.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.964+0000 7f47da06f640 1 -- start start 2024-09-15T21:55:24.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47da06f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4095a60 0x7f47d408bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47da06f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4096540 0x7f47d408c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47da06f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47d412d730 0x7f47d408c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:24.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47da06f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4120110 con 0x7f47d4096540 2024-09-15T21:55:24.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47da06f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4120320 con 0x7f47d4095a60 2024-09-15T21:55:24.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47da06f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40934a0 con 0x7f47d412d730 2024-09-15T21:55:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d37fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4095a60 0x7f47d408bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4096540 0x7f47d408c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d2ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4096540 0x7f47d408c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48038/0 (socket says 172.21.15.47:48038) 2024-09-15T21:55:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d3fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47d412d730 0x7f47d408c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d2ffd640 1 -- 172.21.15.47:0/2601573040 learned_addr learned my addr 172.21.15.47:0/2601573040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d37fe640 1 -- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47d412d730 msgr2=0x7f47d408c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d37fe640 1 --2- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47d412d730 0x7f47d408c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d37fe640 1 -- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4096540 msgr2=0x7f47d408c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:24.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d37fe640 1 --2- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4096540 0x7f47d408c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d37fe640 1 -- 172.21.15.47:0/2601573040 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc009a20 con 0x7f47d4095a60 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d2ffd640 1 --2- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4096540 0x7f47d408c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47d37fe640 1 --2- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4095a60 0x7f47d408bda0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f47c4000c00 tx=0x7f47c4007ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47bb7fe640 1 -- 172.21.15.47:0/2601573040 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40072e0 con 0x7f47d4095a60 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47bb7fe640 1 -- 172.21.15.47:0/2601573040 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c4007460 con 0x7f47d4095a60 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47da06f640 1 -- 172.21.15.47:0/2601573040 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c8009450 con 0x7f47d4095a60 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47bb7fe640 1 -- 172.21.15.47:0/2601573040 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40047a0 con 0x7f47d4095a60 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.965+0000 7f47da06f640 1 -- 172.21.15.47:0/2601573040 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d4094300 con 0x7f47d4095a60 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.967+0000 7f47bb7fe640 1 -- 172.21.15.47:0/2601573040 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f47c4004bb0 con 0x7f47d4095a60 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.967+0000 7f47da06f640 1 -- 172.21.15.47:0/2601573040 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d404bf60 con 0x7f47d4095a60 2024-09-15T21:55:24.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.967+0000 7f47bb7fe640 1 -- 172.21.15.47:0/2601573040 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f47c4062520 con 0x7f47d4095a60 2024-09-15T21:55:24.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:24.970+0000 7f47bb7fe640 1 -- 172.21.15.47:0/2601573040 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f47c403d720 con 0x7f47d4095a60 2024-09-15T21:55:25.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.132+0000 7f47da06f640 1 -- 172.21.15.47:0/2601573040 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f47d412d460 con 0x7f47d4095a60 2024-09-15T21:55:25.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.132+0000 7f47bb7fe640 1 -- 172.21.15.47:0/2601573040 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f47c4040f30 con 0x7f47d4095a60 2024-09-15T21:55:25.135 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:55:25.135 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T21:55:25.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.134+0000 7f47da06f640 1 -- 172.21.15.47:0/2601573040 >> v1:172.21.15.47:6800/32267 conn(0x7f47c004cba0 legacy=0x7f47c004f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:25.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.134+0000 7f47da06f640 1 -- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4095a60 msgr2=0x7f47d408bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:25.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.134+0000 7f47da06f640 1 --2- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4095a60 0x7f47d408bda0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f47c4000c00 tx=0x7f47c4007ca0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.134+0000 7f47da06f640 1 -- 172.21.15.47:0/2601573040 shutdown_connections 2024-09-15T21:55:25.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.134+0000 7f47da06f640 1 --2- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47d412d730 0x7f47d408c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.134+0000 7f47da06f640 1 --2- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47d4096540 0x7f47d408c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.134+0000 7f47da06f640 1 --2- 172.21.15.47:0/2601573040 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47d4095a60 0x7f47d408bda0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.134+0000 7f47da06f640 1 -- 172.21.15.47:0/2601573040 >> 172.21.15.47:0/2601573040 conn(0x7f47d409ac50 msgr2=0x7f47d406c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.134+0000 7f47da06f640 1 -- 172.21.15.47:0/2601573040 shutdown_connections 2024-09-15T21:55:25.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.134+0000 7f47da06f640 1 -- 172.21.15.47:0/2601573040 wait complete. 2024-09-15T21:55:25.147 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T21:55:25.148 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T21:55:25.157 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T21:55:25.157 INFO:teuthology.task.exec:Running commands on role osd.0 host ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:55:25.158 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client quincy' 2024-09-15T21:55:25.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.250+0000 7f7a03c8b640 1 Processor -- start 2024-09-15T21:55:25.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a03c8b640 1 -- start start 2024-09-15T21:55:25.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a03c8b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f79fc068240 0x7f79fc068610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:25.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a03c8b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc068b50 0x7f79fc12e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:25.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a03c8b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc12e900 0x7f79fc130ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a03c8b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc095740 con 0x7f79fc068240 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a03c8b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc06a860 con 0x7f79fc12e900 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a03c8b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc06a280 con 0x7f79fc068b50 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a01a27640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f79fc068240 0x7f79fc068610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a02228640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc12e900 0x7f79fc130ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a01226640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc068b50 0x7f79fc12e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a02228640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc12e900 0x7f79fc130ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52620/0 (socket says 172.21.15.47:52620) 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a02228640 1 -- 172.21.15.47:0/3690542548 learned_addr learned my addr 172.21.15.47:0/3690542548 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a02228640 1 -- 172.21.15.47:0/3690542548 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc068b50 msgr2=0x7f79fc12e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a02228640 1 --2- 172.21.15.47:0/3690542548 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc068b50 0x7f79fc12e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a02228640 1 -- 172.21.15.47:0/3690542548 >> v2:172.21.15.47:3300/0 conn(0x7f79fc068240 msgr2=0x7f79fc068610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.251+0000 7f7a02228640 1 --2- 172.21.15.47:0/3690542548 >> v2:172.21.15.47:3300/0 conn(0x7f79fc068240 0x7f79fc068610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.252+0000 7f7a02228640 1 -- 172.21.15.47:0/3690542548 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc092040 con 0x7f79fc12e900 2024-09-15T21:55:25.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.252+0000 7f7a01226640 1 --2- 172.21.15.47:0/3690542548 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc068b50 0x7f79fc12e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.252+0000 7f7a02228640 1 --2- 172.21.15.47:0/3690542548 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc12e900 0x7f79fc130ce0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009f50 tx=0x7f79f4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c51f5fae0914cba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.252+0000 7f7a00a25640 1 -- 172.21.15.47:0/3690542548 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4031400 con 0x7f79fc12e900 2024-09-15T21:55:25.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.252+0000 7f7a00a25640 1 -- 172.21.15.47:0/3690542548 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f402d050 con 0x7f79fc12e900 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.252+0000 7f7a00a25640 1 -- 172.21.15.47:0/3690542548 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f40317d0 con 0x7f79fc12e900 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.253+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3690542548 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc12e900 msgr2=0x7f79fc130ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.253+0000 7f7a03c8b640 1 --2- 172.21.15.47:0/3690542548 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc12e900 0x7f79fc130ce0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009f50 tx=0x7f79f4009840 comp rx=0 tx=0).stop 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.253+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3690542548 shutdown_connections 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.253+0000 7f7a03c8b640 1 --2- 172.21.15.47:0/3690542548 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc12e900 0x7f79fc130ce0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.253+0000 7f7a03c8b640 1 --2- 172.21.15.47:0/3690542548 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc068b50 0x7f79fc12e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.253+0000 7f7a03c8b640 1 --2- 172.21.15.47:0/3690542548 >> v2:172.21.15.47:3300/0 conn(0x7f79fc068240 0x7f79fc068610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.253+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3690542548 >> 172.21.15.47:0/3690542548 conn(0x7f79fc097d20 msgr2=0x7f79fc098120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.253+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3690542548 shutdown_connections 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.253+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3690542548 wait complete. 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.254+0000 7f7a03c8b640 1 Processor -- start 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.254+0000 7f7a03c8b640 1 -- start start 2024-09-15T21:55:25.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a03c8b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79fc068240 0x7f79fc08f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a03c8b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc068b50 0x7f79fc08fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a03c8b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc12e900 0x7f79fc0901e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a03c8b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0911f0 con 0x7f79fc068240 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a03c8b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc094c30 con 0x7f79fc068b50 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a03c8b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc090810 con 0x7f79fc12e900 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a01a27640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79fc068240 0x7f79fc08f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a01226640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc068b50 0x7f79fc08fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a02228640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc12e900 0x7f79fc0901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a01a27640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79fc068240 0x7f79fc08f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48046/0 (socket says 172.21.15.47:48046) 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a01a27640 1 -- 172.21.15.47:0/3269301081 learned_addr learned my addr 172.21.15.47:0/3269301081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a01226640 1 -- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc12e900 msgr2=0x7f79fc0901e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a01226640 1 --2- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc12e900 0x7f79fc0901e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a01226640 1 -- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79fc068240 msgr2=0x7f79fc08f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a01226640 1 --2- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79fc068240 0x7f79fc08f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a01226640 1 -- 172.21.15.47:0/3269301081 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0009390 con 0x7f79fc068b50 2024-09-15T21:55:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a02228640 1 --2- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc12e900 0x7f79fc0901e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a01226640 1 --2- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc068b50 0x7f79fc08fca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f79f000ea50 tx=0x7f79f00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f79edffb640 1 -- 172.21.15.47:0/3269301081 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0009670 con 0x7f79fc068b50 2024-09-15T21:55:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f79edffb640 1 -- 172.21.15.47:0/3269301081 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f000ca10 con 0x7f79fc068b50 2024-09-15T21:55:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3269301081 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f4009ae0 con 0x7f79fc068b50 2024-09-15T21:55:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.255+0000 7f79edffb640 1 -- 172.21.15.47:0/3269301081 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f000cdb0 con 0x7f79fc068b50 2024-09-15T21:55:25.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.256+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3269301081 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc138e50 con 0x7f79fc068b50 2024-09-15T21:55:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.257+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3269301081 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc071b00 con 0x7f79fc068b50 2024-09-15T21:55:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.257+0000 7f79edffb640 1 -- 172.21.15.47:0/3269301081 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f79f0013680 con 0x7f79fc068b50 2024-09-15T21:55:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.257+0000 7f79edffb640 1 -- 172.21.15.47:0/3269301081 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f79f00634b0 con 0x7f79fc068b50 2024-09-15T21:55:25.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.261+0000 7f79edffb640 1 -- 172.21.15.47:0/3269301081 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f79f003e600 con 0x7f79fc068b50 2024-09-15T21:55:25.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:25.409+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3269301081 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "quincy"} v 0) v1 -- 0x7f79fc063600 con 0x7f79fc068b50 2024-09-15T21:55:26.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.298+0000 7f79edffb640 1 -- 172.21.15.47:0/3269301081 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "quincy"}]=0 set require_min_compat_client to quincy v18) v1 ==== 141+0+0 (secure 0 0 0) 0x7f79f0041e10 con 0x7f79fc068b50 2024-09-15T21:55:26.301 INFO:teuthology.orchestra.run.smithi047.stderr:set require_min_compat_client to quincy 2024-09-15T21:55:26.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.300+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3269301081 >> v1:172.21.15.47:6800/32267 conn(0x7f79e004cb70 legacy=0x7f79e004f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:26.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.300+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc068b50 msgr2=0x7f79fc08fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:26.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.300+0000 7f7a03c8b640 1 --2- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc068b50 0x7f79fc08fca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f79f000ea50 tx=0x7f79f00090f0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.300+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3269301081 shutdown_connections 2024-09-15T21:55:26.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.300+0000 7f7a03c8b640 1 --2- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f79fc12e900 0x7f79fc0901e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.300+0000 7f7a03c8b640 1 --2- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f79fc068b50 0x7f79fc08fca0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.300+0000 7f7a03c8b640 1 --2- 172.21.15.47:0/3269301081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f79fc068240 0x7f79fc08f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.300+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3269301081 >> 172.21.15.47:0/3269301081 conn(0x7f79fc097d20 msgr2=0x7f79fc093d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:26.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.300+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3269301081 shutdown_connections 2024-09-15T21:55:26.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.300+0000 7f7a03c8b640 1 -- 172.21.15.47:0/3269301081 wait complete. 2024-09-15T21:55:26.313 INFO:teuthology.run_tasks:Running task print... 2024-09-15T21:55:26.323 INFO:teuthology.task.print:**** done ceph 2024-09-15T21:55:26.323 INFO:teuthology.run_tasks:Running task print... 2024-09-15T21:55:26.331 INFO:teuthology.task.print:*** upgrading, no cephfs present 2024-09-15T21:55:26.331 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T21:55:26.340 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T21:55:26.340 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:55:26.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs dump' 2024-09-15T21:55:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.443+0000 7f053a35d640 1 Processor -- start 2024-09-15T21:55:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.443+0000 7f053a35d640 1 -- start start 2024-09-15T21:55:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f053a35d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0534096cb0 0x7f053408f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f053a35d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f053412fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f053a35d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0534090e30 con 0x7f053408f8b0 2024-09-15T21:55:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f053a35d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0534090800 con 0x7f0534096cb0 2024-09-15T21:55:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f053a35d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340904d0 con 0x7f053408fce0 2024-09-15T21:55:26.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f0533fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0534096cb0 0x7f053408f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f0533fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0534096cb0 0x7f053408f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48062/0 (socket says 172.21.15.47:48062) 2024-09-15T21:55:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f05337fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f053408f8b0 0x7f053412c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53098/0 (socket says 172.21.15.47:53098) 2024-09-15T21:55:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f0533fff640 1 -- 172.21.15.47:0/3174914272 learned_addr learned my addr 172.21.15.47:0/3174914272 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f05388fa640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f053412fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f0533fff640 1 -- 172.21.15.47:0/3174914272 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 msgr2=0x7f053412fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:55:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f0533fff640 1 --2- 172.21.15.47:0/3174914272 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f053412fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f0533fff640 1 -- 172.21.15.47:0/3174914272 >> v1:172.21.15.47:6789/0 conn(0x7f053408f8b0 legacy=0x7f053412c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f0533fff640 1 -- 172.21.15.47:0/3174914272 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340978b0 con 0x7f0534096cb0 2024-09-15T21:55:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f0532ffd640 1 -- 172.21.15.47:0/3174914272 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433142692 0 0) 0x7f0534090e30 con 0x7f053408f8b0 2024-09-15T21:55:26.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.444+0000 7f05388fa640 1 --2- 172.21.15.47:0/3174914272 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f053412fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.445+0000 7f0533fff640 1 --2- 172.21.15.47:0/3174914272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0534096cb0 0x7f053408f370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0524009910 tx=0x7f05240091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a91c5f6867222b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.445+0000 7f0532ffd640 1 -- 172.21.15.47:0/3174914272 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052402ad20 con 0x7f0534096cb0 2024-09-15T21:55:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.445+0000 7f0532ffd640 1 -- 172.21.15.47:0/3174914272 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052402ae80 con 0x7f0534096cb0 2024-09-15T21:55:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.445+0000 7f0532ffd640 1 -- 172.21.15.47:0/3174914272 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524032670 con 0x7f0534096cb0 2024-09-15T21:55:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.445+0000 7f053a35d640 1 -- 172.21.15.47:0/3174914272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0534096cb0 msgr2=0x7f053408f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.445+0000 7f053a35d640 1 --2- 172.21.15.47:0/3174914272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0534096cb0 0x7f053408f370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0524009910 tx=0x7f05240091d0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.446+0000 7f053a35d640 1 -- 172.21.15.47:0/3174914272 shutdown_connections 2024-09-15T21:55:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.446+0000 7f053a35d640 1 --2- 172.21.15.47:0/3174914272 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f053412fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.446+0000 7f053a35d640 1 --2- 172.21.15.47:0/3174914272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0534096cb0 0x7f053408f370 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.446+0000 7f053a35d640 1 -- 172.21.15.47:0/3174914272 >> 172.21.15.47:0/3174914272 conn(0x7f053409be90 msgr2=0x7f0534097da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.446+0000 7f053a35d640 1 -- 172.21.15.47:0/3174914272 shutdown_connections 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.446+0000 7f053a35d640 1 -- 172.21.15.47:0/3174914272 wait complete. 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.446+0000 7f053a35d640 1 Processor -- start 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.446+0000 7f053a35d640 1 -- start start 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f053a35d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053408f8b0 0x7f053412fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f053a35d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f0534068270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f053a35d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0534096cb0 0x7f05340687b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f053a35d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053408d440 con 0x7f053408f8b0 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f053a35d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0534069cd0 con 0x7f053408fce0 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f053a35d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f053408d000 con 0x7f0534096cb0 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f0533fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053408f8b0 0x7f053412fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:26.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f05337fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f0534068270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f05388fa640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0534096cb0 0x7f05340687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f05337fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f0534068270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:52638/0 (socket says 172.21.15.47:52638) 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f0533fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053408f8b0 0x7f053412fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48078/0 (socket says 172.21.15.47:48078) 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f05337fe640 1 -- 172.21.15.47:0/3989369314 learned_addr learned my addr 172.21.15.47:0/3989369314 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f05388fa640 1 -- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 msgr2=0x7f0534068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f05388fa640 1 --2- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f0534068270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f05388fa640 1 -- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053408f8b0 msgr2=0x7f053412fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f05388fa640 1 --2- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053408f8b0 0x7f053412fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f05388fa640 1 -- 172.21.15.47:0/3989369314 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c009a20 con 0x7f0534096cb0 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.447+0000 7f05337fe640 1 --2- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f0534068270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.448+0000 7f05388fa640 1 --2- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0534096cb0 0x7f05340687b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f052c009f10 tx=0x7f052c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:55:26.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.448+0000 7f051bfff640 1 -- 172.21.15.47:0/3989369314 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c00cdb0 con 0x7f0534096cb0 2024-09-15T21:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.448+0000 7f051bfff640 1 -- 172.21.15.47:0/3989369314 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c00f040 con 0x7f0534096cb0 2024-09-15T21:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.448+0000 7f053a35d640 1 -- 172.21.15.47:0/3989369314 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0524009450 con 0x7f0534096cb0 2024-09-15T21:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.448+0000 7f051bfff640 1 -- 172.21.15.47:0/3989369314 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0146d0 con 0x7f0534096cb0 2024-09-15T21:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.448+0000 7f053a35d640 1 -- 172.21.15.47:0/3989369314 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053406a690 con 0x7f0534096cb0 2024-09-15T21:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.449+0000 7f053a35d640 1 -- 172.21.15.47:0/3989369314 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05340917f0 con 0x7f0534096cb0 2024-09-15T21:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.449+0000 7f051bfff640 1 -- 172.21.15.47:0/3989369314 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f052c003020 con 0x7f0534096cb0 2024-09-15T21:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.449+0000 7f051bfff640 1 -- 172.21.15.47:0/3989369314 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2808+0+0 (secure 0 0 0) 0x7f052c063b90 con 0x7f0534096cb0 2024-09-15T21:55:26.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.453+0000 7f051bfff640 1 -- 172.21.15.47:0/3989369314 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f052c03ed00 con 0x7f0534096cb0 2024-09-15T21:55:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.610+0000 7f053a35d640 1 -- 172.21.15.47:0/3989369314 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f053408c7d0 con 0x7f0534096cb0 2024-09-15T21:55:26.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.611+0000 7f051bfff640 1 -- 172.21.15.47:0/3989369314 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 1 v1) v1 ==== 75+0+358 (secure 0 0 0) 0x7f052c018070 con 0x7f0534096cb0 2024-09-15T21:55:26.614 INFO:teuthology.orchestra.run.smithi047.stdout:e1 2024-09-15T21:55:26.614 INFO:teuthology.orchestra.run.smithi047.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T21:55:26.614 INFO:teuthology.orchestra.run.smithi047.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-15T21:55:26.614 INFO:teuthology.orchestra.run.smithi047.stdout:legacy client fscid: -1 2024-09-15T21:55:26.614 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:55:26.614 INFO:teuthology.orchestra.run.smithi047.stdout:No filesystems configured 2024-09-15T21:55:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 1 2024-09-15T21:55:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.612+0000 7f053a35d640 1 -- 172.21.15.47:0/3989369314 >> v1:172.21.15.47:6800/32267 conn(0x7f052004cb70 legacy=0x7f052004f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.612+0000 7f053a35d640 1 -- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0534096cb0 msgr2=0x7f05340687b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:55:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.612+0000 7f053a35d640 1 --2- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0534096cb0 0x7f05340687b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f052c009f10 tx=0x7f052c00b040 comp rx=0 tx=0).stop 2024-09-15T21:55:26.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.612+0000 7f053a35d640 1 -- 172.21.15.47:0/3989369314 shutdown_connections 2024-09-15T21:55:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.612+0000 7f053a35d640 1 --2- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f0534096cb0 0x7f05340687b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.612+0000 7f053a35d640 1 --2- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f053408fce0 0x7f0534068270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.612+0000 7f053a35d640 1 --2- 172.21.15.47:0/3989369314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f053408f8b0 0x7f053412fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:55:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.612+0000 7f053a35d640 1 -- 172.21.15.47:0/3989369314 >> 172.21.15.47:0/3989369314 conn(0x7f053409be90 msgr2=0x7f053409a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:55:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.613+0000 7f053a35d640 1 -- 172.21.15.47:0/3989369314 shutdown_connections 2024-09-15T21:55:26.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:55:26.613+0000 7f053a35d640 1 -- 172.21.15.47:0/3989369314 wait complete. 2024-09-15T21:55:26.625 INFO:teuthology.run_tasks:Running task install.upgrade... 2024-09-15T21:55:26.634 INFO:teuthology.task.install:project ceph config {'mon.a': None} overrides {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-15T21:55:26.634 INFO:teuthology.task.install:remote ubuntu@smithi047.front.sepia.ceph.com config {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-15T21:55:26.634 INFO:teuthology.task.install:extra packages: [] 2024-09-15T21:55:26.639 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T21:55:26.640 INFO:teuthology.task.install:Upgrading ceph rpm packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd 2024-09-15T21:55:26.640 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph-common --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:55:26.661 INFO:teuthology.orchestra.run.smithi047.stdout:17.2.7-1633.g6d025bc2.el9 2024-09-15T21:55:26.661 INFO:teuthology.packaging:The installed version of ceph-common is 17.2.7-1633.g6d025bc2.el9 2024-09-15T21:55:26.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:55:26.825 INFO:teuthology.task.install:Ceph rpm upgrade from 17.2.7-1633.g6d025bc2.el9 to 19.1.1-326.g8921c8ef 2024-09-15T21:55:26.825 INFO:teuthology.task.install.rpm:Host smithi047 is: centos 9.stream x86_64 2024-09-15T21:55:26.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:55:26.962 INFO:teuthology.task.install.rpm:Repo base URL: https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T21:55:26.962 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/yum.repos.d/ceph.repo 2024-09-15T21:55:27.098 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:55:27.098 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-15T21:55:27.098 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:55:27.127 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:55:27.195 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:55:27.261 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:55:27.318 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-09-15T21:55:27.330 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-09-15T21:55:27.571 INFO:teuthology.orchestra.run.smithi047.stdout:60 files removed 2024-09-15T21:55:27.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y upgrade ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd 2024-09-15T21:55:28.181 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 303 kB/s | 83 kB 00:00 2024-09-15T21:55:28.549 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-09-15T21:55:28.816 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-15T21:55:29.516 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-15T21:55:34.402 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 20 MB 00:03 2024-09-15T21:55:38.969 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-15T21:55:40.437 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 19 kB 00:00 2024-09-15T21:55:41.508 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-15T21:55:47.156 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-15T21:55:49.496 INFO:teuthology.orchestra.run.smithi047.stderr:Package cephadm available, but not installed. 2024-09-15T21:55:49.496 INFO:teuthology.orchestra.run.smithi047.stdout:No match for argument: cephadm 2024-09-15T21:55:49.502 INFO:teuthology.orchestra.run.smithi047.stderr:Package ceph-mgr-dashboard available, but not installed. 2024-09-15T21:55:49.502 INFO:teuthology.orchestra.run.smithi047.stdout:No match for argument: ceph-mgr-dashboard 2024-09-15T21:55:49.503 INFO:teuthology.orchestra.run.smithi047.stderr:Package ceph-mgr-diskprediction-local available, but not installed. 2024-09-15T21:55:49.503 INFO:teuthology.orchestra.run.smithi047.stdout:No match for argument: ceph-mgr-diskprediction-local 2024-09-15T21:55:49.503 INFO:teuthology.orchestra.run.smithi047.stderr:Package ceph-mgr-rook available, but not installed. 2024-09-15T21:55:49.503 INFO:teuthology.orchestra.run.smithi047.stdout:No match for argument: ceph-mgr-rook 2024-09-15T21:55:49.504 INFO:teuthology.orchestra.run.smithi047.stderr:Package ceph-mgr-cephadm available, but not installed. 2024-09-15T21:55:49.504 INFO:teuthology.orchestra.run.smithi047.stdout:No match for argument: ceph-mgr-cephadm 2024-09-15T21:55:49.507 INFO:teuthology.orchestra.run.smithi047.stderr:Package ceph-volume available, but not installed. 2024-09-15T21:55:49.507 INFO:teuthology.orchestra.run.smithi047.stdout:No match for argument: ceph-volume 2024-09-15T21:55:49.603 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-15T21:55:49.606 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:55:49.607 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-15T21:55:49.607 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:55:49.607 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-09-15T21:55:49.607 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T21:55:49.607 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T21:55:49.607 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T21:55:49.607 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T21:55:49.607 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache 2024-09-15T21:55:49.607 INFO:teuthology.orchestra.run.smithi047.stdout: x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T21:55:49.608 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T21:55:49.609 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T21:55:49.610 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:55:49.611 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-15T21:55:49.611 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-09-15T21:55:49.611 INFO:teuthology.orchestra.run.smithi047.stdout:Install 11 Packages 2024-09-15T21:55:49.611 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 30 Packages 2024-09-15T21:55:49.611 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:55:49.611 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 137 M 2024-09-15T21:55:49.611 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-15T21:55:50.557 INFO:teuthology.orchestra.run.smithi047.stdout:(1/41): boost-program-options-1.75.0-8.el9.x86_ 155 kB/s | 107 kB 00:00 2024-09-15T21:55:50.608 INFO:teuthology.orchestra.run.smithi047.stdout:(2/41): lua-5.4.4-4.el9.x86_64.rpm 254 kB/s | 188 kB 00:00 2024-09-15T21:55:50.675 INFO:teuthology.orchestra.run.smithi047.stdout:(3/41): qatlib-24.02.0-1.el9.x86_64.rpm 273 kB/s | 221 kB 00:00 2024-09-15T21:55:50.758 INFO:teuthology.orchestra.run.smithi047.stdout:(4/41): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-09-15T21:55:51.059 INFO:teuthology.orchestra.run.smithi047.stdout:(5/41): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-15T21:55:51.084 INFO:teuthology.orchestra.run.smithi047.stdout:(6/41): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-15T21:55:51.109 INFO:teuthology.orchestra.run.smithi047.stdout:(7/41): qatlib-service-24.02.0-1.el9.x86_64.rpm 65 kB/s | 36 kB 00:00 2024-09-15T21:55:51.134 INFO:teuthology.orchestra.run.smithi047.stdout:(8/41): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-15T21:55:51.185 INFO:teuthology.orchestra.run.smithi047.stdout:(9/41): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-15T21:55:51.210 INFO:teuthology.orchestra.run.smithi047.stdout:(10/41): qatzip-libs-1.2.0-1.el9.x86_64.rpm 79 kB/s | 47 kB 00:00 2024-09-15T21:55:51.235 INFO:teuthology.orchestra.run.smithi047.stdout:(11/41): re2-20211101-20.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-15T21:55:51.336 INFO:teuthology.orchestra.run.smithi047.stdout:(12/41): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 44 kB/s | 6.5 kB 00:00 2024-09-15T21:55:51.577 INFO:teuthology.orchestra.run.smithi047.stdout:(13/41): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.6 MB/s | 896 kB 00:00 2024-09-15T21:55:51.670 INFO:teuthology.orchestra.run.smithi047.stdout:(14/41): ceph-base-19.1.1-326.g8921c8ef.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-15T21:55:51.694 INFO:teuthology.orchestra.run.smithi047.stdout:(15/41): ceph-immutable-object-cache-19.1.1-326 1.2 MB/s | 145 kB 00:00 2024-09-15T21:55:51.812 INFO:teuthology.orchestra.run.smithi047.stdout:(16/41): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 17 MB/s | 2.4 MB 00:00 2024-09-15T21:55:51.937 INFO:teuthology.orchestra.run.smithi047.stdout:(17/41): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 6.3 MB/s | 1.5 MB 00:00 2024-09-15T21:55:52.096 INFO:teuthology.orchestra.run.smithi047.stdout:(18/41): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-15T21:55:52.571 INFO:teuthology.orchestra.run.smithi047.stdout:(19/41): ceph-common-19.1.1-326.g8921c8ef.el9.x 16 MB/s | 21 MB 00:01 2024-09-15T21:55:52.705 INFO:teuthology.orchestra.run.smithi047.stdout:(20/41): ceph-radosgw-19.1.1-326.g8921c8ef.el9. 17 MB/s | 11 MB 00:00 2024-09-15T21:55:52.733 INFO:teuthology.orchestra.run.smithi047.stdout:(21/41): ceph-selinux-19.1.1-326.g8921c8ef.el9. 158 kB/s | 25 kB 00:00 2024-09-15T21:55:52.814 INFO:teuthology.orchestra.run.smithi047.stdout:(22/41): libcephfs-devel-19.1.1-326.g8921c8ef.e 380 kB/s | 32 kB 00:00 2024-09-15T21:55:52.931 INFO:teuthology.orchestra.run.smithi047.stdout:(23/41): libcephfs2-19.1.1-326.g8921c8ef.el9.x8 6.3 MB/s | 751 kB 00:00 2024-09-15T21:55:53.039 INFO:teuthology.orchestra.run.smithi047.stdout:(24/41): libcephsqlite-19.1.1-326.g8921c8ef.el9 1.5 MB/s | 163 kB 00:00 2024-09-15T21:55:53.123 INFO:teuthology.orchestra.run.smithi047.stdout:(25/41): librados-devel-19.1.1-326.g8921c8ef.el 1.5 MB/s | 127 kB 00:00 2024-09-15T21:55:53.465 INFO:teuthology.orchestra.run.smithi047.stdout:(26/41): ceph-osd-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 17 MB 00:01 2024-09-15T21:55:53.568 INFO:teuthology.orchestra.run.smithi047.stdout:(27/41): librados2-19.1.1-326.g8921c8ef.el9.x86 7.7 MB/s | 3.4 MB 00:00 2024-09-15T21:55:53.607 INFO:teuthology.orchestra.run.smithi047.stdout:(28/41): libradosstriper1-19.1.1-326.g8921c8ef. 3.5 MB/s | 506 kB 00:00 2024-09-15T21:55:53.749 INFO:teuthology.orchestra.run.smithi047.stdout:(29/41): librbd1-19.1.1-326.g8921c8ef.el9.x86_6 17 MB/s | 3.2 MB 00:00 2024-09-15T21:55:53.816 INFO:teuthology.orchestra.run.smithi047.stdout:(30/41): python3-ceph-argparse-19.1.1-326.g8921 677 kB/s | 45 kB 00:00 2024-09-15T21:55:53.900 INFO:teuthology.orchestra.run.smithi047.stdout:(31/41): python3-ceph-common-19.1.1-326.g8921c8 1.6 MB/s | 136 kB 00:00 2024-09-15T21:55:54.017 INFO:teuthology.orchestra.run.smithi047.stdout:(32/41): librgw2-19.1.1-326.g8921c8ef.el9.x86_6 13 MB/s | 5.3 MB 00:00 2024-09-15T21:55:54.066 INFO:teuthology.orchestra.run.smithi047.stdout:(33/41): python3-cephfs-19.1.1-326.g8921c8ef.el 972 kB/s | 161 kB 00:00 2024-09-15T21:55:54.093 INFO:teuthology.orchestra.run.smithi047.stdout:(34/41): python3-rados-19.1.1-326.g8921c8ef.el9 4.1 MB/s | 320 kB 00:00 2024-09-15T21:55:54.126 INFO:teuthology.orchestra.run.smithi047.stdout:(35/41): python3-rbd-19.1.1-326.g8921c8ef.el9.x 4.9 MB/s | 300 kB 00:00 2024-09-15T21:55:54.151 INFO:teuthology.orchestra.run.smithi047.stdout:(36/41): python3-rgw-19.1.1-326.g8921c8ef.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-15T21:55:54.177 INFO:teuthology.orchestra.run.smithi047.stdout:(37/41): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86_ 1.7 MB/s | 85 kB 00:00 2024-09-15T21:55:54.235 INFO:teuthology.orchestra.run.smithi047.stdout:(38/41): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_6 2.9 MB/s | 170 kB 00:00 2024-09-15T21:55:54.702 INFO:teuthology.orchestra.run.smithi047.stdout:(39/41): ceph-test-19.1.1-326.g8921c8ef.el9.x86 24 MB/s | 49 MB 00:01 2024-09-15T21:55:54.728 INFO:teuthology.orchestra.run.smithi047.stdout:(40/41): ceph-mgr-modules-core-19.1.1-326.g8921 507 kB/s | 249 kB 00:00 2024-09-15T21:55:54.778 INFO:teuthology.orchestra.run.smithi047.stdout:(41/41): rbd-mirror-19.1.1-326.g8921c8ef.el9.x8 5.0 MB/s | 3.1 MB 00:00 2024-09-15T21:55:54.783 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:55:54.784 INFO:teuthology.orchestra.run.smithi047.stdout:Total 26 MB/s | 137 MB 00:05 2024-09-15T21:55:54.858 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-15T21:55:54.923 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-15T21:55:54.924 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-15T21:55:55.991 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-15T21:55:55.992 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-15T21:55:59.104 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-15T21:55:59.271 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 1/71 2024-09-15T21:55:59.633 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 1/71 2024-09-15T21:55:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2/71 2024-09-15T21:55:59.839 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2/71 2024-09-15T21:55:59.890 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 3/71 2024-09-15T21:55:59.966 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 3/71 2024-09-15T21:56:00.031 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 4/71 2024-09-15T21:56:00.066 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 5/71 2024-09-15T21:56:00.156 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 5/71 2024-09-15T21:56:00.193 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 6/71 2024-09-15T21:56:00.256 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 6/71 2024-09-15T21:56:00.294 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 7/71 2024-09-15T21:56:00.312 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 7/71 2024-09-15T21:56:00.455 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 8/71 2024-09-15T21:56:00.488 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 8/71 2024-09-15T21:56:00.532 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x 9/71 2024-09-15T21:56:00.568 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 10/71 2024-09-15T21:56:00.642 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 11/71 2024-09-15T21:56:00.692 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-5.el9.noarch 12/71 2024-09-15T21:56:00.738 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 13/71 2024-09-15T21:56:00.798 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/71 2024-09-15T21:56:00.857 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 15/71 2024-09-15T21:56:01.020 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 16/71 2024-09-15T21:56:01.144 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.n 17/71 2024-09-15T21:56:01.201 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86 18/71 2024-09-15T21:56:01.325 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/71 2024-09-15T21:56:01.374 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/71 2024-09-15T21:56:01.409 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 21/71 2024-09-15T21:56:01.446 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/71 2024-09-15T21:56:02.061 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/71 2024-09-15T21:56:02.124 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 24/71 2024-09-15T21:56:02.196 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 24/71 2024-09-15T21:56:02.234 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 25/71 2024-09-15T21:56:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 26/71 2024-09-15T21:56:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 26/71 2024-09-15T21:56:05.661 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 26/71 2024-09-15T21:56:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 27/71 2024-09-15T21:56:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 27/71 2024-09-15T21:56:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 28/71 2024-09-15T21:56:15.114 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 28/71 2024-09-15T21:56:15.206 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 29/71 2024-09-15T21:56:15.383 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 29/71 2024-09-15T21:56:15.423 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 30/71 2024-09-15T21:56:15.961 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 30/71 2024-09-15T21:56:16.017 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 31/71 2024-09-15T21:56:17.882 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 31/71 2024-09-15T21:56:17.936 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 32/71 2024-09-15T21:56:17.949 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 32/71 2024-09-15T21:56:19.121 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 33/71 2024-09-15T21:56:19.221 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 34/71 2024-09-15T21:56:19.243 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 34/71 2024-09-15T21:56:19.288 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef 35/71 2024-09-15T21:56:19.660 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef 35/71 2024-09-15T21:56:19.705 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 36/71 2024-09-15T21:56:25.626 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 36/71 2024-09-15T21:56:25.870 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 37/71 2024-09-15T21:56:25.914 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 38/71 2024-09-15T21:56:25.965 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 39/71 2024-09-15T21:56:26.091 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 40/71 2024-09-15T21:56:26.142 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 41/71 2024-09-15T21:56:26.175 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 42/71 2024-09-15T21:56:26.221 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 43/71 2024-09-15T21:56:26.229 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 44/71 2024-09-15T21:56:26.229 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 44/71 2024-09-15T21:56:26.314 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 44/71 2024-09-15T21:56:26.356 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 45/71 2024-09-15T21:56:26.363 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 46/71 2024-09-15T21:56:26.363 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 46/71 2024-09-15T21:56:26.398 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 46/71 2024-09-15T21:56:26.404 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 47/71 2024-09-15T21:56:26.404 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 47/71 2024-09-15T21:56:26.448 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 47/71 2024-09-15T21:56:26.473 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 48/71 2024-09-15T21:56:26.478 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc 49/71 2024-09-15T21:56:26.478 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc 49/71 2024-09-15T21:56:26.507 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc 49/71 2024-09-15T21:56:26.532 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 50/71 2024-09-15T21:56:26.533 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 51/71 2024-09-15T21:56:26.600 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 51/71 2024-09-15T21:56:26.625 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 52/71 2024-09-15T21:56:26.630 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 53/71 2024-09-15T21:56:26.630 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 53/71 2024-09-15T21:56:26.659 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 53/71 2024-09-15T21:56:26.664 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 54/71 2024-09-15T21:56:26.664 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 54/71 2024-09-15T21:56:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 54/71 2024-09-15T21:56:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 55/71 2024-09-15T21:56:26.715 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 55/71 2024-09-15T21:56:26.752 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 55/71 2024-09-15T21:56:26.752 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 56/71 2024-09-15T21:56:26.785 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 56/71 2024-09-15T21:56:26.793 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 57/71 2024-09-15T21:56:26.793 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 57/71 2024-09-15T21:56:26.889 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 57/71 2024-09-15T21:56:26.890 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 58/71 2024-09-15T21:56:26.923 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 58/71 2024-09-15T21:56:26.923 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 59/71 2024-09-15T21:56:27.007 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 59/71 2024-09-15T21:56:27.032 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 60/71 2024-09-15T21:56:27.065 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 61/71 2024-09-15T21:56:27.065 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 62/71 2024-09-15T21:56:27.159 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 62/71 2024-09-15T21:56:27.183 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 63/71 2024-09-15T21:56:27.183 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 64/71 2024-09-15T21:56:27.268 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 64/71 2024-09-15T21:56:27.293 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 65/71 2024-09-15T21:56:27.293 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 66/71 2024-09-15T21:56:27.377 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 66/71 2024-09-15T21:56:27.402 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 67/71 2024-09-15T21:56:27.427 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9. 68/71 2024-09-15T21:56:27.453 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x8 69/71 2024-09-15T21:56:27.487 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9. 70/71 2024-09-15T21:56:27.487 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 71/71 2024-09-15T21:56:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 71/71 2024-09-15T21:56:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 1/71 2024-09-15T21:56:29.039 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 2/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 3/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 4/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 5/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 6/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 7/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 8/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 9/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 10/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-20.el9.x86_64 11/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 12/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:17.2.7-1633.g6d025bc2.el9.x86_64 13/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 14/71 2024-09-15T21:56:29.040 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:17.2.7-1633.g6d025bc2.el9.x86_64 15/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 16/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:17.2.7-1633.g6d025bc2.el9.x86_64 17/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 18/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 19/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef 20/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1633.g6d025bc 21/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 22/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:17.2.7-1633.g6d025bc2.el9.x86_64 23/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 24/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:17.2.7-1633.g6d025bc2.el9.x86_64 25/71 2024-09-15T21:56:29.041 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 26/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:17.2.7-1633.g6d025bc2.el9.x86_64 27/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 28/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:17.2.7-1633.g6d025bc2.el9.x86_64 29/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 30/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 31/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 32/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:17.2.7-1633.g6d025bc2.el9.x86_64 33/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 34/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:17.2.7-1633.g6d025bc2.el9.x86_64 35/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 36/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 37/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 38/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:17.2.7-1633.g6d025bc2.el9.x86_64 39/71 2024-09-15T21:56:29.042 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 40/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:17.2.7-1633.g6d025bc2.el9.x86_64 41/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 42/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:17.2.7-1633.g6d025bc2.el9.x86_64 43/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 44/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:17.2.7-1633.g6d025bc2.el9.x86_64 45/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 46/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:17.2.7-1633.g6d025bc2.el9.x86_6 47/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 48/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:17.2.7-1633.g6d025bc2.el9.x86_64 49/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 50/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:17.2.7-1633.g6d025bc2.el9.x86_64 51/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x 52/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1633.g6d025bc2.el9. 53/71 2024-09-15T21:56:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86 54/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:17.2.7-1633.g6d025bc2.el9.x8 55/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 56/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:17.2.7-1633.g6d025bc2.el9.x86_64 57/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 58/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:17.2.7-1633.g6d025bc2.el9.x86_64 59/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 60/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 61/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 62/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:17.2.7-1633.g6d025bc2.el9.x86_64 63/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 64/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:17.2.7-1633.g6d025bc2.el9.x86_64 65/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 66/71 2024-09-15T21:56:29.044 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:17.2.7-1633.g6d025bc2.el9.x86_64 67/71 2024-09-15T21:56:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 68/71 2024-09-15T21:56:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:17.2.7-1633.g6d025bc2.el9.x86_64 69/71 2024-09-15T21:56:29.045 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.n 70/71 2024-09-15T21:56:30.213 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1633.g6d025bc2.el9. 71/71 2024-09-15T21:56:30.213 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:30.213 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-09-15T21:56:30.213 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.214 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.215 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 libarrow-9.0.0-11.el9.x86_64 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-11.el9.noarch lua-5.4.4-4.el9.x86_64 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 luarocks-3.9.2-5.el9.noarch 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-11.el9.x86_64 qatlib-24.02.0-1.el9.x86_64 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-24.02.0-1.el9.x86_64 qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:30.216 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-15T21:56:30.542 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:56:30.689 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:56:30.712 INFO:teuthology.orchestra.run.smithi047.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T21:56:30.712 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T21:56:30.712 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T21:56:30.714 INFO:teuthology.run_tasks:Running task print... 2024-09-15T21:56:30.724 INFO:teuthology.task.print:**** done install.upgrade 2024-09-15T21:56:30.724 INFO:teuthology.run_tasks:Running task ceph.restart... 2024-09-15T21:56:30.743 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:56:30.744 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:56:30.765 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:56:30.765 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:56:30.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:56:30.807 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:56:30.808 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:56:30.808 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:56:30.821 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:56:30.821 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:56:30.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:56:30.864 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:56:30.864 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:56:30.864 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:56:30.876 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:56:30.876 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:56:30.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:56:30.919 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:56:30.919 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T21:56:30.919 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:56:30.942 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T21:56:30.943 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T21:56:30.943 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T21:56:30.985 INFO:tasks.ceph.mgr.x:Started 2024-09-15T21:56:30.985 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-15T21:56:30.985 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:56:31.007 INFO:tasks.ceph.mgr.y:Stopped 2024-09-15T21:56:31.007 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-15T21:56:31.007 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-15T21:56:31.050 INFO:tasks.ceph.mgr.y:Started 2024-09-15T21:56:31.051 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-09-15T21:56:31.060 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T21:56:31.060 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T21:56:31.060 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T21:56:31.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.207+0000 7f1a5ceaa640 1 Processor -- start 2024-09-15T21:56:31.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a5ceaa640 1 -- start start 2024-09-15T21:56:31.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a5ceaa640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812f3c0 0x7f1a5812f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a5ceaa640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a5812fce0 0x7f1a58098090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a5ceaa640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a580985d0 0x7f1a58098a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a5ceaa640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1a5806f260 con 0x7f1a580985d0 2024-09-15T21:56:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a5ceaa640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f1a58099600 con 0x7f1a5812f3c0 2024-09-15T21:56:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a5ceaa640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f1a5809f230 con 0x7f1a5812fce0 2024-09-15T21:56:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a55d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a5812fce0 0x7f1a58098090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a5659c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812f3c0 0x7f1a5812f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a5659c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812f3c0 0x7f1a5812f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39710/0 (socket says 172.21.15.47:39710) 2024-09-15T21:56:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.208+0000 7f1a55d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a5812fce0 0x7f1a58098090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46254/0 (socket says 172.21.15.47:46254) 2024-09-15T21:56:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.209+0000 7f1a5659c640 1 -- 172.21.15.47:0/848064491 learned_addr learned my addr 172.21.15.47:0/848064491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.209+0000 7f1a56d9d640 1 --2- 172.21.15.47:0/848064491 >> v2:172.21.15.47:3300/0 conn(0x7f1a580985d0 0x7f1a58098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.209+0000 7f1a55d9b640 1 -- 172.21.15.47:0/848064491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812f3c0 msgr2=0x7f1a5812f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:31.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.209+0000 7f1a55d9b640 1 --2- 172.21.15.47:0/848064491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812f3c0 0x7f1a5812f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.209+0000 7f1a55d9b640 1 -- 172.21.15.47:0/848064491 >> v2:172.21.15.47:3300/0 conn(0x7f1a580985d0 msgr2=0x7f1a58098a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:31.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.209+0000 7f1a55d9b640 1 --2- 172.21.15.47:0/848064491 >> v2:172.21.15.47:3300/0 conn(0x7f1a580985d0 0x7f1a58098a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.209+0000 7f1a55d9b640 1 -- 172.21.15.47:0/848064491 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a58130120 con 0x7f1a5812fce0 2024-09-15T21:56:31.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.210+0000 7f1a55d9b640 1 --2- 172.21.15.47:0/848064491 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a5812fce0 0x7f1a58098090 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0097a0 tx=0x7f1a4c029fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50d12032dfeec13d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:31.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.210+0000 7f1a5559a640 1 -- 172.21.15.47:0/848064491 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c037070 con 0x7f1a5812fce0 2024-09-15T21:56:31.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.210+0000 7f1a5559a640 1 -- 172.21.15.47:0/848064491 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a4c02ad20 con 0x7f1a5812fce0 2024-09-15T21:56:31.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.210+0000 7f1a5559a640 1 -- 172.21.15.47:0/848064491 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c032660 con 0x7f1a5812fce0 2024-09-15T21:56:31.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.211+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/848064491 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a5812fce0 msgr2=0x7f1a58098090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.211+0000 7f1a5ceaa640 1 --2- 172.21.15.47:0/848064491 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a5812fce0 0x7f1a58098090 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0097a0 tx=0x7f1a4c029fd0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.211+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/848064491 shutdown_connections 2024-09-15T21:56:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.211+0000 7f1a5ceaa640 1 --2- 172.21.15.47:0/848064491 >> v2:172.21.15.47:3300/0 conn(0x7f1a580985d0 0x7f1a58098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.211+0000 7f1a5ceaa640 1 --2- 172.21.15.47:0/848064491 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a5812fce0 0x7f1a58098090 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.211+0000 7f1a5ceaa640 1 --2- 172.21.15.47:0/848064491 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812f3c0 0x7f1a5812f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.211+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/848064491 >> 172.21.15.47:0/848064491 conn(0x7f1a580a23b0 msgr2=0x7f1a5809e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.211+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/848064491 shutdown_connections 2024-09-15T21:56:31.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.211+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/848064491 wait complete. 2024-09-15T21:56:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.213+0000 7f1a5ceaa640 1 Processor -- start 2024-09-15T21:56:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.213+0000 7f1a5ceaa640 1 -- start start 2024-09-15T21:56:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.213+0000 7f1a5ceaa640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a580985d0 0x7f1a581270e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.213+0000 7f1a5ceaa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a5812f3c0 0x7f1a58127620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.213+0000 7f1a5ceaa640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812fce0 0x7f1a58127b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.213+0000 7f1a5ceaa640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58136920 con 0x7f1a5812f3c0 2024-09-15T21:56:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.213+0000 7f1a5ceaa640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f1a5806ee60 con 0x7f1a5812fce0 2024-09-15T21:56:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.213+0000 7f1a5ceaa640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f1a58136aa0 con 0x7f1a580985d0 2024-09-15T21:56:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a5659c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a580985d0 0x7f1a581270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a56d9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812fce0 0x7f1a58127b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a56d9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812fce0 0x7f1a58127b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39722/0 (socket says 172.21.15.47:39722) 2024-09-15T21:56:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a5659c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a580985d0 0x7f1a581270e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46270/0 (socket says 172.21.15.47:46270) 2024-09-15T21:56:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a56d9d640 1 -- 172.21.15.47:0/3072168469 learned_addr learned my addr 172.21.15.47:0/3072168469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:31.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a55d9b640 1 --2- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a5812f3c0 0x7f1a58127620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a56d9d640 1 -- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a580985d0 msgr2=0x7f1a581270e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a56d9d640 1 --2- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a580985d0 0x7f1a581270e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a56d9d640 1 -- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a5812f3c0 msgr2=0x7f1a58127620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a56d9d640 1 --2- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a5812f3c0 0x7f1a58127620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a56d9d640 1 -- 172.21.15.47:0/3072168469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a581286c0 con 0x7f1a5812fce0 2024-09-15T21:56:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.214+0000 7f1a55d9b640 1 --2- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a5812f3c0 0x7f1a58127620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.215+0000 7f1a56d9d640 1 --2- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812fce0 0x7f1a58127b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000ef00 tx=0x7f1a5000b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.215+0000 7f1a46ffd640 1 -- 172.21.15.47:0/3072168469 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50018020 con 0x7f1a5812fce0 2024-09-15T21:56:31.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.215+0000 7f1a46ffd640 1 -- 172.21.15.47:0/3072168469 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a50014400 con 0x7f1a5812fce0 2024-09-15T21:56:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.215+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/3072168469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a581280a0 con 0x7f1a5812fce0 2024-09-15T21:56:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.215+0000 7f1a46ffd640 1 -- 172.21.15.47:0/3072168469 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500147f0 con 0x7f1a5812fce0 2024-09-15T21:56:31.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.215+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/3072168469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a58129980 con 0x7f1a5812fce0 2024-09-15T21:56:31.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.216+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/3072168469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a5812f7a0 con 0x7f1a5812fce0 2024-09-15T21:56:31.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.217+0000 7f1a46ffd640 1 -- 172.21.15.47:0/3072168469 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7f1a50005d10 con 0x7f1a5812fce0 2024-09-15T21:56:31.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.217+0000 7f1a46ffd640 1 -- 172.21.15.47:0/3072168469 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7f1a50063da0 con 0x7f1a5812fce0 2024-09-15T21:56:31.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.217+0000 7f1a5659c640 1 -- 172.21.15.47:0/3072168469 >> v1:172.21.15.47:6800/32267 conn(0x7f1a3804d080 legacy=0x7f1a3804f540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:31.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.225+0000 7f1a46ffd640 1 -- 172.21.15.47:0/3072168469 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7f1a50022070 con 0x7f1a5812fce0 2024-09-15T21:56:31.263 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:31.261+0000 7f2e5472f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T21:56:31.264 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:31.263+0000 7fd4a305c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T21:56:31.307 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:31.305+0000 7f2e5472f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T21:56:31.307 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:31.306+0000 7fd4a305c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T21:56:31.361 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:31.360+0000 7f2e5472f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T21:56:31.362 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:31.361+0000 7fd4a305c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T21:56:31.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.402+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/3072168469 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1a58068e40 con 0x7f1a5812fce0 2024-09-15T21:56:31.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.405+0000 7f1a46ffd640 1 -- 172.21.15.47:0/3072168469 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v6) ==== 74+0+116441 (secure 0 0 0) 0x7f1a5003ef30 con 0x7f1a5812fce0 2024-09-15T21:56:31.408 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.408+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/3072168469 >> v1:172.21.15.47:6800/32267 conn(0x7f1a3804d080 legacy=0x7f1a3804f540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.408+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812fce0 msgr2=0x7f1a58127b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.408+0000 7f1a5ceaa640 1 --2- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812fce0 0x7f1a58127b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000ef00 tx=0x7f1a5000b480 comp rx=0 tx=0).stop 2024-09-15T21:56:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.408+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/3072168469 shutdown_connections 2024-09-15T21:56:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.408+0000 7f1a5ceaa640 1 --2- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1a5812fce0 0x7f1a58127b60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.408+0000 7f1a5ceaa640 1 --2- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a5812f3c0 0x7f1a58127620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.408+0000 7f1a5ceaa640 1 --2- 172.21.15.47:0/3072168469 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1a580985d0 0x7f1a581270e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.408+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/3072168469 >> 172.21.15.47:0/3072168469 conn(0x7f1a580a23b0 msgr2=0x7f1a5812e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.408+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/3072168469 shutdown_connections 2024-09-15T21:56:31.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.408+0000 7f1a5ceaa640 1 -- 172.21.15.47:0/3072168469 wait complete. 2024-09-15T21:56:31.425 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":6,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6800","nonce":32267}]},"active_addr":"172.21.15.47:6800/32267","active_change":"2024-09-15T21:55:02.690184+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4106,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3200219561}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1981014788}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":1739125583}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2444109955}]}]} 2024-09-15T21:56:31.428 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T21:56:31.428 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:56:31.428 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:56:31.492 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:31.491+0000 7f2e5472f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:56:31.493 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:31.491+0000 7fd4a305c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:56:31.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.531+0000 7f50c6f3e640 1 Processor -- start 2024-09-15T21:56:31.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50c6f3e640 1 -- start start 2024-09-15T21:56:31.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50c6f3e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50c006cd30 0x7f50c006ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50c6f3e640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c006f390 0x7f50c006f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50c6f3e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50c0070530 con 0x7f50c0133ca0 2024-09-15T21:56:31.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50c6f3e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f50c009d8e0 con 0x7f50c006cd30 2024-09-15T21:56:31.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50c6f3e640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f50c00707d0 con 0x7f50c006f390 2024-09-15T21:56:31.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50bffff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c006f390 0x7f50c006f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50c4cda640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50c006cd30 0x7f50c006ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50bffff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c006f390 0x7f50c006f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46272/0 (socket says 172.21.15.47:46272) 2024-09-15T21:56:31.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50c4cda640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50c006cd30 0x7f50c006ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47476/0 (socket says 172.21.15.47:47476) 2024-09-15T21:56:31.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.532+0000 7f50bffff640 1 -- 172.21.15.47:0/2995514542 learned_addr learned my addr 172.21.15.47:0/2995514542 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:31.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.533+0000 7f50c4cda640 1 -- 172.21.15.47:0/2995514542 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c006f390 msgr2=0x7f50c006f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:31.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.533+0000 7f50c4cda640 1 --2- 172.21.15.47:0/2995514542 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c006f390 0x7f50c006f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.533+0000 7f50c4cda640 1 -- 172.21.15.47:0/2995514542 >> v1:172.21.15.47:6789/0 conn(0x7f50c0133ca0 legacy=0x7f50c0136090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.533+0000 7f50c4cda640 1 -- 172.21.15.47:0/2995514542 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c006db70 con 0x7f50c006cd30 2024-09-15T21:56:31.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.533+0000 7f50bffff640 1 --2- 172.21.15.47:0/2995514542 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c006f390 0x7f50c006f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:31.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.534+0000 7f50c4cda640 1 --2- 172.21.15.47:0/2995514542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 0x7f50c006ee50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f50b40097d0 tx=0x7f50b402a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4504096783fbfbcf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:31.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.534+0000 7f50bf7fe640 1 -- 172.21.15.47:0/2995514542 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4037070 con 0x7f50c006cd30 2024-09-15T21:56:31.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.534+0000 7f50bf7fe640 1 -- 172.21.15.47:0/2995514542 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b402adb0 con 0x7f50c006cd30 2024-09-15T21:56:31.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.534+0000 7f50bf7fe640 1 -- 172.21.15.47:0/2995514542 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4032660 con 0x7f50c006cd30 2024-09-15T21:56:31.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.535+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2995514542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 msgr2=0x7f50c006ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.535+0000 7f50c6f3e640 1 --2- 172.21.15.47:0/2995514542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 0x7f50c006ee50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f50b40097d0 tx=0x7f50b402a0a0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.535+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2995514542 shutdown_connections 2024-09-15T21:56:31.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.535+0000 7f50c6f3e640 1 --2- 172.21.15.47:0/2995514542 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c006f390 0x7f50c006f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.535+0000 7f50c6f3e640 1 --2- 172.21.15.47:0/2995514542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 0x7f50c006ee50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.535+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2995514542 >> 172.21.15.47:0/2995514542 conn(0x7f50c009b600 msgr2=0x7f50c009ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:31.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.535+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2995514542 shutdown_connections 2024-09-15T21:56:31.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.535+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2995514542 wait complete. 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.536+0000 7f50c6f3e640 1 Processor -- start 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.536+0000 7f50c6f3e640 1 -- start start 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c6f3e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 0x7f50c0095f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c6f3e640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f50c006f390 0x7f50c0096470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c6f3e640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c0133ca0 0x7f50c00969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c6f3e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f50c0137930 con 0x7f50c006cd30 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c6f3e640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f50c01377b0 con 0x7f50c006f390 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c6f3e640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f50c0137ab0 con 0x7f50c0133ca0 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c4cda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 0x7f50c0095f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c54db640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c0133ca0 0x7f50c00969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c4cda640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 0x7f50c0095f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47482/0 (socket says 172.21.15.47:47482) 2024-09-15T21:56:31.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c4cda640 1 -- 172.21.15.47:0/2296396358 learned_addr learned my addr 172.21.15.47:0/2296396358 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50bffff640 1 --2- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f50c006f390 0x7f50c0096470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c4cda640 1 -- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c0133ca0 msgr2=0x7f50c00969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.537+0000 7f50c4cda640 1 --2- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c0133ca0 0x7f50c00969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.538+0000 7f50c4cda640 1 -- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f50c006f390 msgr2=0x7f50c0096470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.538+0000 7f50c4cda640 1 --2- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f50c006f390 0x7f50c0096470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.538+0000 7f50c4cda640 1 -- 172.21.15.47:0/2296396358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c0124be0 con 0x7f50c006cd30 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.538+0000 7f50bffff640 1 --2- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f50c006f390 0x7f50c0096470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.538+0000 7f50c4cda640 1 --2- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 0x7f50c0095f30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50b4035040 tx=0x7f50b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.538+0000 7f50bd7fa640 1 -- 172.21.15.47:0/2296396358 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b404d070 con 0x7f50c006cd30 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.538+0000 7f50bd7fa640 1 -- 172.21.15.47:0/2296396358 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b402abb0 con 0x7f50c006cd30 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.538+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2296396358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c0096ef0 con 0x7f50c006cd30 2024-09-15T21:56:31.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.538+0000 7f50bd7fa640 1 -- 172.21.15.47:0/2296396358 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4032d80 con 0x7f50c006cd30 2024-09-15T21:56:31.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.538+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2296396358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c0097fb0 con 0x7f50c006cd30 2024-09-15T21:56:31.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.539+0000 7f50bd7fa640 1 -- 172.21.15.47:0/2296396358 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7f50b4050070 con 0x7f50c006cd30 2024-09-15T21:56:31.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.539+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2296396358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c0133a90 con 0x7f50c006cd30 2024-09-15T21:56:31.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.540+0000 7f50bffff640 1 -- 172.21.15.47:0/2296396358 >> v1:172.21.15.47:6800/32267 conn(0x7f50a404d080 legacy=0x7f50a404f540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:31.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.540+0000 7f50bd7fa640 1 -- 172.21.15.47:0/2296396358 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7f50b408ba00 con 0x7f50c006cd30 2024-09-15T21:56:31.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.544+0000 7f50bd7fa640 1 -- 172.21.15.47:0/2296396358 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7f50c0133a90 con 0x7f50c006cd30 2024-09-15T21:56:31.613 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:31.612+0000 7fd4a305c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T21:56:31.614 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:31.612+0000 7f2e5472f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T21:56:31.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.685+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2296396358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50c00750e0 con 0x7f50c006cd30 2024-09-15T21:56:31.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.686+0000 7f50bd7fa640 1 -- 172.21.15.47:0/2296396358 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+8667 (secure 0 0 0) 0x7f50b40591f0 con 0x7f50c006cd30 2024-09-15T21:56:31.687 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:31.687 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:55:26.294576+0000","last_up_change":"2024-09-15T21:55:10.248544+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":32760}]},"public_addr":"172.21.15.47:6801/32760","cluster_addr":"172.21.15.47:6802/32760","heartbeat_back_addr":"172.21.15.47:6804/32760","heartbeat_front_addr":"172.21.15.47:6803/32760","state":["exists","up"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":32761}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":32761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":32761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":32761}]},"public_addr":"172.21.15.47:6805/32761","cluster_addr":"172.21.15.47:6806/32761","heartbeat_back_addr":"172.21.15.47:6808/32761","heartbeat_front_addr":"172.21.15.47:6807/32761","state":["exists","up"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":32772}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":32772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":32772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":32772}]},"public_addr":"172.21.15.47:6813/32772","cluster_addr":"172.21.15.47:6814/32772","heartbeat_back_addr":"172.21.15.47:6816/32772","heartbeat_front_addr":"172.21.15.47:6815/32772","state":["exists","up"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":32770}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":32770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":32770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":32770}]},"public_addr":"172.21.15.47:6809/32770","cluster_addr":"172.21.15.47:6810/32770","heartbeat_back_addr":"172.21.15.47:6812/32770","heartbeat_front_addr":"172.21.15.47:6811/32770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:56:31.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.687+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2296396358 >> v1:172.21.15.47:6800/32267 conn(0x7f50a404d080 legacy=0x7f50a404f540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:31.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.688+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 msgr2=0x7f50c0095f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.688+0000 7f50c6f3e640 1 --2- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 0x7f50c0095f30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50b4035040 tx=0x7f50b4004730 comp rx=0 tx=0).stop 2024-09-15T21:56:31.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.688+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2296396358 shutdown_connections 2024-09-15T21:56:31.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.688+0000 7f50c6f3e640 1 --2- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f50c0133ca0 0x7f50c00969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.688+0000 7f50c6f3e640 1 --2- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f50c006f390 0x7f50c0096470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.688+0000 7f50c6f3e640 1 --2- 172.21.15.47:0/2296396358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50c006cd30 0x7f50c0095f30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.688+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2296396358 >> 172.21.15.47:0/2296396358 conn(0x7f50c009b600 msgr2=0x7f50c0073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:31.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.688+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2296396358 shutdown_connections 2024-09-15T21:56:31.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.688+0000 7f50c6f3e640 1 -- 172.21.15.47:0/2296396358 wait complete. 2024-09-15T21:56:31.701 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:56:31.701 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:56:31.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.803+0000 7f71afaaa640 1 Processor -- start 2024-09-15T21:56:31.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.803+0000 7f71afaaa640 1 -- start start 2024-09-15T21:56:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71afaaa640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71a806e2d0 0x7f71a806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71afaaa640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 0x7f71a812c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71afaaa640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a80715d0 con 0x7f71a812c650 2024-09-15T21:56:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71ad846640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71a806e2d0 0x7f71a806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71afaaa640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f71a8070a20 con 0x7f71a806e2d0 2024-09-15T21:56:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71afaaa640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f71a8070400 con 0x7f71a806ebf0 2024-09-15T21:56:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71ad846640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71a806e2d0 0x7f71a806e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47496/0 (socket says 172.21.15.47:47496) 2024-09-15T21:56:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71ad846640 1 -- 172.21.15.47:0/2288651173 learned_addr learned my addr 172.21.15.47:0/2288651173 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:31.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71ad045640 1 --2- 172.21.15.47:0/2288651173 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 0x7f71a812c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71ad846640 1 -- 172.21.15.47:0/2288651173 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 msgr2=0x7f71a812c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:31.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71ad846640 1 --2- 172.21.15.47:0/2288651173 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 0x7f71a812c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71ad846640 1 -- 172.21.15.47:0/2288651173 >> v1:172.21.15.47:6789/0 conn(0x7f71a812c650 legacy=0x7f71a812cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.804+0000 7f71ad846640 1 -- 172.21.15.47:0/2288651173 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a806f070 con 0x7f71a806e2d0 2024-09-15T21:56:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.805+0000 7f71ad846640 1 --2- 172.21.15.47:0/2288651173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71a806e2d0 0x7f71a806e6b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009870 tx=0x7f71a402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcea4ee7c226aba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.805+0000 7f71ac844640 1 -- 172.21.15.47:0/2288651173 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4037070 con 0x7f71a806e2d0 2024-09-15T21:56:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.805+0000 7f71ac844640 1 -- 172.21.15.47:0/2288651173 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a4032440 con 0x7f71a806e2d0 2024-09-15T21:56:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.805+0000 7f71ac844640 1 -- 172.21.15.47:0/2288651173 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4032810 con 0x7f71a806e2d0 2024-09-15T21:56:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.806+0000 7f71afaaa640 1 -- 172.21.15.47:0/2288651173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71a806e2d0 msgr2=0x7f71a806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.806+0000 7f71afaaa640 1 --2- 172.21.15.47:0/2288651173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71a806e2d0 0x7f71a806e6b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009870 tx=0x7f71a402a130 comp rx=0 tx=0).stop 2024-09-15T21:56:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.806+0000 7f71afaaa640 1 -- 172.21.15.47:0/2288651173 shutdown_connections 2024-09-15T21:56:31.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.806+0000 7f71afaaa640 1 --2- 172.21.15.47:0/2288651173 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 0x7f71a812c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.806+0000 7f71afaaa640 1 --2- 172.21.15.47:0/2288651173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71a806e2d0 0x7f71a806e6b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.806+0000 7f71afaaa640 1 -- 172.21.15.47:0/2288651173 >> 172.21.15.47:0/2288651173 conn(0x7f71a809ef20 msgr2=0x7f71a809f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:31.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.806+0000 7f71afaaa640 1 -- 172.21.15.47:0/2288651173 shutdown_connections 2024-09-15T21:56:31.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.806+0000 7f71afaaa640 1 -- 172.21.15.47:0/2288651173 wait complete. 2024-09-15T21:56:31.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.807+0000 7f71afaaa640 1 Processor -- start 2024-09-15T21:56:31.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.807+0000 7f71afaaa640 1 -- start start 2024-09-15T21:56:31.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.807+0000 7f71afaaa640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71a806e2d0 0x7f71a8129f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.807+0000 7f71afaaa640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 0x7f71a812a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.807+0000 7f71afaaa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71a812c650 0x7f71a812a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:31.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71afaaa640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f71a81347e0 con 0x7f71a812c650 2024-09-15T21:56:31.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71afaaa640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f71a8134660 con 0x7f71a806ebf0 2024-09-15T21:56:31.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71afaaa640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f71a8134960 con 0x7f71a806e2d0 2024-09-15T21:56:31.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ad045640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 0x7f71a812a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ad045640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 0x7f71a812a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39748/0 (socket says 172.21.15.47:39748) 2024-09-15T21:56:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ad045640 1 -- 172.21.15.47:0/833720197 learned_addr learned my addr 172.21.15.47:0/833720197 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ad846640 1 --2- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71a806e2d0 0x7f71a8129f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ae047640 1 --2- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71a812c650 0x7f71a812a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ad045640 1 -- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71a806e2d0 msgr2=0x7f71a8129f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ad045640 1 --2- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71a806e2d0 0x7f71a8129f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ad045640 1 -- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71a812c650 msgr2=0x7f71a812a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ad045640 1 --2- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71a812c650 0x7f71a812a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ad045640 1 -- 172.21.15.47:0/833720197 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a812b960 con 0x7f71a806ebf0 2024-09-15T21:56:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f71ad045640 1 --2- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 0x7f71a812a450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f719800e9b0 tx=0x7f719800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.808+0000 7f719e7fc640 1 -- 172.21.15.47:0/833720197 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198017070 con 0x7f71a806ebf0 2024-09-15T21:56:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.809+0000 7f719e7fc640 1 -- 172.21.15.47:0/833720197 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f719800c9a0 con 0x7f71a806ebf0 2024-09-15T21:56:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.809+0000 7f719e7fc640 1 -- 172.21.15.47:0/833720197 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719800cd90 con 0x7f71a806ebf0 2024-09-15T21:56:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.809+0000 7f71afaaa640 1 -- 172.21.15.47:0/833720197 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a8133390 con 0x7f71a806ebf0 2024-09-15T21:56:31.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.809+0000 7f71afaaa640 1 -- 172.21.15.47:0/833720197 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a812b1b0 con 0x7f71a806ebf0 2024-09-15T21:56:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.810+0000 7f719e7fc640 1 -- 172.21.15.47:0/833720197 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7f719801a070 con 0x7f71a806ebf0 2024-09-15T21:56:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.810+0000 7f71afaaa640 1 -- 172.21.15.47:0/833720197 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a806e6b0 con 0x7f71a806ebf0 2024-09-15T21:56:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.810+0000 7f71ad846640 1 -- 172.21.15.47:0/833720197 >> v1:172.21.15.47:6800/32267 conn(0x7f718c04d050 legacy=0x7f718c04f510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:31.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.810+0000 7f719e7fc640 1 -- 172.21.15.47:0/833720197 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7f71980637d0 con 0x7f71a806ebf0 2024-09-15T21:56:31.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.814+0000 7f719e7fc640 1 -- 172.21.15.47:0/833720197 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7f71a812b960 con 0x7f71a806ebf0 2024-09-15T21:56:31.883 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:31.881+0000 7fd4a305c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T21:56:31.883 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:31.881+0000 7f2e5472f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T21:56:31.933 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:31.931+0000 7f2e5472f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T21:56:31.933 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:31.931+0000 7fd4a305c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T21:56:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.967+0000 7f71afaaa640 1 -- 172.21.15.47:0/833720197 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71a8068e40 con 0x7f71a806ebf0 2024-09-15T21:56:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.968+0000 7f719e7fc640 1 -- 172.21.15.47:0/833720197 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+8667 (secure 0 0 0) 0x7f719803ea00 con 0x7f71a806ebf0 2024-09-15T21:56:31.970 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:31.970 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":18,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:55:26.294576+0000","last_up_change":"2024-09-15T21:55:10.248544+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":32760}]},"public_addr":"172.21.15.47:6801/32760","cluster_addr":"172.21.15.47:6802/32760","heartbeat_back_addr":"172.21.15.47:6804/32760","heartbeat_front_addr":"172.21.15.47:6803/32760","state":["exists","up"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":32761}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":32761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":32761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":32761}]},"public_addr":"172.21.15.47:6805/32761","cluster_addr":"172.21.15.47:6806/32761","heartbeat_back_addr":"172.21.15.47:6808/32761","heartbeat_front_addr":"172.21.15.47:6807/32761","state":["exists","up"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":32772}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":32772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":32772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":32772}]},"public_addr":"172.21.15.47:6813/32772","cluster_addr":"172.21.15.47:6814/32772","heartbeat_back_addr":"172.21.15.47:6816/32772","heartbeat_front_addr":"172.21.15.47:6815/32772","state":["exists","up"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":32770}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":32770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":32770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":32770}]},"public_addr":"172.21.15.47:6809/32770","cluster_addr":"172.21.15.47:6810/32770","heartbeat_back_addr":"172.21.15.47:6812/32770","heartbeat_front_addr":"172.21.15.47:6811/32770","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:56:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.970+0000 7f71afaaa640 1 -- 172.21.15.47:0/833720197 >> v1:172.21.15.47:6800/32267 conn(0x7f718c04d050 legacy=0x7f718c04f510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.970+0000 7f71afaaa640 1 -- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 msgr2=0x7f71a812a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.970+0000 7f71afaaa640 1 --2- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 0x7f71a812a450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f719800e9b0 tx=0x7f719800ee70 comp rx=0 tx=0).stop 2024-09-15T21:56:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.970+0000 7f71afaaa640 1 -- 172.21.15.47:0/833720197 shutdown_connections 2024-09-15T21:56:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.970+0000 7f71afaaa640 1 --2- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71a812c650 0x7f71a812a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.970+0000 7f71afaaa640 1 --2- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f71a806ebf0 0x7f71a812a450 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.970+0000 7f71afaaa640 1 --2- 172.21.15.47:0/833720197 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f71a806e2d0 0x7f71a8129f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.970+0000 7f71afaaa640 1 -- 172.21.15.47:0/833720197 >> 172.21.15.47:0/833720197 conn(0x7f71a809ef20 msgr2=0x7f71a806d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.970+0000 7f71afaaa640 1 -- 172.21.15.47:0/833720197 shutdown_connections 2024-09-15T21:56:31.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:31.970+0000 7f71afaaa640 1 -- 172.21.15.47:0/833720197 wait complete. 2024-09-15T21:56:31.983 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:56:31.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:56:31.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:56:31.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T21:56:32.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.100+0000 7f8d78a44640 1 Processor -- start 2024-09-15T21:56:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.100+0000 7f8d78a44640 1 -- start start 2024-09-15T21:56:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d78a44640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d7406e740 0x7f8d7406eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d78a44640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d7412bab0 0x7f8d7412beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d78a44640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d7412c3f0 0x7f8d74132de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d78a44640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8d74090570 con 0x7f8d7412bab0 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d78a44640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7409f460 con 0x7f8d7412c3f0 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d71d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d7412bab0 0x7f8d7412beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d71d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8d7412bab0 0x7f8d7412beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47510/0 (socket says 172.21.15.47:47510) 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d71d9b640 1 -- 172.21.15.47:0/634639194 learned_addr learned my addr 172.21.15.47:0/634639194 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d78a44640 1 -- 172.21.15.47:0/634639194 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f8d7406e2f0 con 0x7f8d7406e740 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d7259c640 1 --2- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d7406e740 0x7f8d7406eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.101+0000 7f8d72d9d640 1 --2- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d7412c3f0 0x7f8d74132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.102+0000 7f8d71d9b640 1 -- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d7406e740 msgr2=0x7f8d7406eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.102+0000 7f8d71d9b640 1 --2- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d7406e740 0x7f8d7406eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.102+0000 7f8d71d9b640 1 -- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d7412c3f0 msgr2=0x7f8d74132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.102+0000 7f8d71d9b640 1 --2- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d7412c3f0 0x7f8d74132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.102+0000 7f8d71d9b640 1 -- 172.21.15.47:0/634639194 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7406fa60 con 0x7f8d7412bab0 2024-09-15T21:56:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.102+0000 7f8d72d9d640 1 --2- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d7412c3f0 0x7f8d74132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.103+0000 7f8d71d9b640 1 --2- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7412bab0 0x7f8d7412beb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d680098a0 tx=0x7f8d6802a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1de0a0169693c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.104+0000 7f8d7159a640 1 -- 172.21.15.47:0/634639194 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68037070 con 0x7f8d7412bab0 2024-09-15T21:56:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.104+0000 7f8d7159a640 1 -- 172.21.15.47:0/634639194 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6802add0 con 0x7f8d7412bab0 2024-09-15T21:56:32.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.104+0000 7f8d7159a640 1 -- 172.21.15.47:0/634639194 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68032660 con 0x7f8d7412bab0 2024-09-15T21:56:32.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.104+0000 7f8d78a44640 1 -- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7412bab0 msgr2=0x7f8d7412beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.104+0000 7f8d78a44640 1 --2- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7412bab0 0x7f8d7412beb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d680098a0 tx=0x7f8d6802a0c0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.105+0000 7fc62335e640 1 Processor -- start 2024-09-15T21:56:32.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.105+0000 7f8d78a44640 1 -- 172.21.15.47:0/634639194 shutdown_connections 2024-09-15T21:56:32.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.105+0000 7f8d78a44640 1 --2- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d7412c3f0 0x7f8d74132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.105+0000 7f8d78a44640 1 --2- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7412bab0 0x7f8d7412beb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.105+0000 7f8d78a44640 1 --2- 172.21.15.47:0/634639194 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d7406e740 0x7f8d7406eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.105+0000 7f8d78a44640 1 -- 172.21.15.47:0/634639194 >> 172.21.15.47:0/634639194 conn(0x7f8d7409e5f0 msgr2=0x7f8d7409ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.105+0000 7f8d78a44640 1 -- 172.21.15.47:0/634639194 shutdown_connections 2024-09-15T21:56:32.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.105+0000 7f8d78a44640 1 -- 172.21.15.47:0/634639194 wait complete. 2024-09-15T21:56:32.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.105+0000 7fc62335e640 1 -- start start 2024-09-15T21:56:32.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.105+0000 7fc62335e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc61c132940 0x7fc61c134d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7fc62335e640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc61c135270 0x7fc61c137660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7f8d78a44640 1 Processor -- start 2024-09-15T21:56:32.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7fc6208f9640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc61c132940 0x7fc61c134d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7fc6210fa640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc61c095e60 0x7fc61c096240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36788/0 (socket says 172.21.15.47:36788) 2024-09-15T21:56:32.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7fc6210fa640 1 -- 172.21.15.47:0/3322921302 learned_addr learned my addr 172.21.15.47:0/3322921302 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:32.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7fc6218fb640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc61c135270 0x7fc61c137660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7fc62335e640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61c091b90 con 0x7fc61c095e60 2024-09-15T21:56:32.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7f8d78a44640 1 -- start start 2024-09-15T21:56:32.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7fc62335e640 1 -- 172.21.15.47:0/3322921302 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc61c095a50 con 0x7fc61c132940 2024-09-15T21:56:32.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7fc62335e640 1 -- 172.21.15.47:0/3322921302 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fc61c068410 con 0x7fc61c135270 2024-09-15T21:56:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.106+0000 7f8d78a44640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d7406e740 0x7f8d7409b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7f8d78a44640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d7412bab0 0x7f8d7409b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7f8d78a44640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7412c3f0 0x7f8d74127680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7fc613fff640 1 -- 172.21.15.47:0/3322921302 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 503374663 0 0) 0x7fc61c091b90 con 0x7fc61c095e60 2024-09-15T21:56:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7fc6208f9640 1 -- 172.21.15.47:0/3322921302 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc61c135270 msgr2=0x7fc61c137660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7fc6208f9640 1 --2- 172.21.15.47:0/3322921302 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc61c135270 0x7fc61c137660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7f8d71d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d7412bab0 0x7f8d7409b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7f8d71d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d7412bab0 0x7f8d7409b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46292/0 (socket says 172.21.15.47:46292) 2024-09-15T21:56:32.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7fc6208f9640 1 -- 172.21.15.47:0/3322921302 >> v1:172.21.15.47:6789/0 conn(0x7fc61c095e60 legacy=0x7fc61c096240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7f8d71d9b640 1 -- 172.21.15.47:0/3993285149 learned_addr learned my addr 172.21.15.47:0/3993285149 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7f8d72d9d640 1 --2- 172.21.15.47:0/3993285149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7412c3f0 0x7f8d74127680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7fc6208f9640 1 -- 172.21.15.47:0/3322921302 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c096ca0 con 0x7fc61c132940 2024-09-15T21:56:32.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7406f690 con 0x7f8d7412c3f0 2024-09-15T21:56:32.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f8d740901e0 con 0x7f8d7406e740 2024-09-15T21:56:32.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f8d7406f810 con 0x7f8d7412bab0 2024-09-15T21:56:32.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.107+0000 7f8d7259c640 1 --2- 172.21.15.47:0/3993285149 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d7406e740 0x7f8d7409b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d72d9d640 1 -- 172.21.15.47:0/3993285149 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d7412bab0 msgr2=0x7f8d7409b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d72d9d640 1 --2- 172.21.15.47:0/3993285149 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f8d7412bab0 0x7f8d7409b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d72d9d640 1 -- 172.21.15.47:0/3993285149 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d7406e740 msgr2=0x7f8d7409b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d72d9d640 1 --2- 172.21.15.47:0/3993285149 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f8d7406e740 0x7f8d7409b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7fc6208f9640 1 --2- 172.21.15.47:0/3322921302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc61c132940 0x7fc61c134d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0097a0 tx=0x7fc60c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924192bad50c7c9f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d72d9d640 1 -- 172.21.15.47:0/3993285149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d741290f0 con 0x7f8d7412c3f0 2024-09-15T21:56:32.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d72d9d640 1 --2- 172.21.15.47:0/3993285149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7412c3f0 0x7f8d74127680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009d90 tx=0x7f8d6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d62ffd640 1 -- 172.21.15.47:0/3993285149 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c018030 con 0x7f8d7412c3f0 2024-09-15T21:56:32.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d62ffd640 1 -- 172.21.15.47:0/3993285149 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c00cd30 con 0x7f8d7412c3f0 2024-09-15T21:56:32.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d62ffd640 1 -- 172.21.15.47:0/3993285149 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c003bd0 con 0x7f8d7412c3f0 2024-09-15T21:56:32.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d74073f80 con 0x7f8d7412c3f0 2024-09-15T21:56:32.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.109+0000 7fc613fff640 1 -- 172.21.15.47:0/3322921302 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c037070 con 0x7fc61c132940 2024-09-15T21:56:32.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.109+0000 7fc613fff640 1 -- 172.21.15.47:0/3322921302 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c02adb0 con 0x7fc61c132940 2024-09-15T21:56:32.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.109+0000 7fc613fff640 1 -- 172.21.15.47:0/3322921302 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c037070 con 0x7fc61c132940 2024-09-15T21:56:32.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.108+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7412d730 con 0x7f8d7412c3f0 2024-09-15T21:56:32.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7f8d62ffd640 1 -- 172.21.15.47:0/3993285149 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7f8d6c005d10 con 0x7f8d7412c3f0 2024-09-15T21:56:32.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d7409f6e0 con 0x7f8d7412c3f0 2024-09-15T21:56:32.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7fc62335e640 1 -- 172.21.15.47:0/3322921302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc61c132940 msgr2=0x7fc61c134d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7fc62335e640 1 --2- 172.21.15.47:0/3322921302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc61c132940 0x7fc61c134d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0097a0 tx=0x7fc60c02a060 comp rx=0 tx=0).stop 2024-09-15T21:56:32.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7fc62335e640 1 -- 172.21.15.47:0/3322921302 shutdown_connections 2024-09-15T21:56:32.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7fc62335e640 1 --2- 172.21.15.47:0/3322921302 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc61c135270 0x7fc61c137660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7fc62335e640 1 --2- 172.21.15.47:0/3322921302 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc61c132940 0x7fc61c134d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7fc62335e640 1 -- 172.21.15.47:0/3322921302 >> 172.21.15.47:0/3322921302 conn(0x7fc61c09c310 msgr2=0x7fc61c09c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7f8d62ffd640 1 -- 172.21.15.47:0/3993285149 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7f8d6c063f80 con 0x7f8d7412c3f0 2024-09-15T21:56:32.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7f8d7259c640 1 -- 172.21.15.47:0/3993285149 >> v1:172.21.15.47:6800/32267 conn(0x7f8d5404d120 legacy=0x7f8d5404f5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7f8d62ffd640 1 -- 172.21.15.47:0/3993285149 --> v1:172.21.15.47:6805/32761 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d6c017e50 con 0x7f8d54053810 2024-09-15T21:56:32.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.110+0000 7f8d62ffd640 1 -- 172.21.15.47:0/3993285149 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8d6c069570 con 0x7f8d7412c3f0 2024-09-15T21:56:32.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.111+0000 7fc62335e640 1 -- 172.21.15.47:0/3322921302 shutdown_connections 2024-09-15T21:56:32.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.111+0000 7fc62335e640 1 -- 172.21.15.47:0/3322921302 wait complete. 2024-09-15T21:56:32.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7f8d62ffd640 1 -- 172.21.15.47:0/3993285149 <== osd.1 v1:172.21.15.47:6805/32761 1 ==== command_reply(tid 1: 0 ) ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f8d6c017e50 con 0x7f8d54053810 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc62335e640 1 Processor -- start 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc62335e640 1 -- start start 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc62335e640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc61c095e60 0x7fc61c12dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc62335e640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc61c132940 0x7fc61c12e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc62335e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc61c135270 0x7fc61c12e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc62335e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c137ba0 con 0x7fc61c135270 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc62335e640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fc61c091840 con 0x7fc61c132940 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc62335e640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fc61c137d20 con 0x7fc61c095e60 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc6218fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc61c135270 0x7fc61c12e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc6208f9640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc61c132940 0x7fc61c12e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc6208f9640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc61c132940 0x7fc61c12e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39774/0 (socket says 172.21.15.47:39774) 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.113+0000 7fc6208f9640 1 -- 172.21.15.47:0/1106466613 learned_addr learned my addr 172.21.15.47:0/1106466613 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.112+0000 7fc6210fa640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc61c095e60 0x7fc61c12dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.113+0000 7fc6218fb640 1 -- 172.21.15.47:0/1106466613 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc61c095e60 msgr2=0x7fc61c12dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.113+0000 7fc6218fb640 1 --2- 172.21.15.47:0/1106466613 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc61c095e60 0x7fc61c12dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.113+0000 7fc6218fb640 1 -- 172.21.15.47:0/1106466613 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc61c132940 msgr2=0x7fc61c12e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.113+0000 7fc6218fb640 1 --2- 172.21.15.47:0/1106466613 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc61c132940 0x7fc61c12e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.113+0000 7fc6218fb640 1 -- 172.21.15.47:0/1106466613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c06e1e0 con 0x7fc61c135270 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.113+0000 7fc6218fb640 1 --2- 172.21.15.47:0/1106466613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc61c135270 0x7fc61c12e7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0953f0 tx=0x7fc61800d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.114+0000 7fc611ffb640 1 -- 172.21.15.47:0/1106466613 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6180125f0 con 0x7fc61c135270 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.114+0000 7fc611ffb640 1 -- 172.21.15.47:0/1106466613 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc618019070 con 0x7fc61c135270 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.114+0000 7fc611ffb640 1 -- 172.21.15.47:0/1106466613 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6180129b0 con 0x7fc61c135270 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.114+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61c130330 con 0x7fc61c135270 2024-09-15T21:56:32.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.114+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61c12ff60 con 0x7fc61c135270 2024-09-15T21:56:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.115+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc61c096240 con 0x7fc61c135270 2024-09-15T21:56:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.116+0000 7fc611ffb640 1 -- 172.21.15.47:0/1106466613 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7fc618012b50 con 0x7fc61c135270 2024-09-15T21:56:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.117+0000 7fc611ffb640 1 -- 172.21.15.47:0/1106466613 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7fc618063af0 con 0x7fc61c135270 2024-09-15T21:56:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.117+0000 7fc6210fa640 1 -- 172.21.15.47:0/1106466613 >> v1:172.21.15.47:6800/32267 conn(0x7fc60004d120 legacy=0x7fc60004f5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.117+0000 7fc611ffb640 1 -- 172.21.15.47:0/1106466613 --> v1:172.21.15.47:6809/32770 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6000574c0 con 0x7fc600053810 2024-09-15T21:56:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.118+0000 7fc611ffb640 1 -- 172.21.15.47:0/1106466613 <== osd.3 v1:172.21.15.47:6809/32770 1 ==== command_reply(tid 1: 0 ) ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fc6000574c0 con 0x7fc600053810 2024-09-15T21:56:32.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.121+0000 7fc611ffb640 1 -- 172.21.15.47:0/1106466613 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fc61803e550 con 0x7fc61c135270 2024-09-15T21:56:32.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.136+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 --> v1:172.21.15.47:6805/32761 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d7408f550 con 0x7f8d54053810 2024-09-15T21:56:32.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.136+0000 7f8d62ffd640 1 -- 172.21.15.47:0/3993285149 <== osd.1 v1:172.21.15.47:6805/32761 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 3077695196) 0x7f8d7408f550 con 0x7f8d54053810 2024-09-15T21:56:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.137+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 >> v1:172.21.15.47:6805/32761 conn(0x7f8d54053810 legacy=0x7f8d54055c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.137+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 >> v1:172.21.15.47:6800/32267 conn(0x7f8d5404d120 legacy=0x7f8d5404f5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.137+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7412c3f0 msgr2=0x7f8d74127680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.137+0000 7f8d78a44640 1 --2- 172.21.15.47:0/3993285149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d7412c3f0 0x7f8d74127680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009d90 tx=0x7f8d6c00c490 comp rx=0 tx=0).stop 2024-09-15T21:56:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.137+0000 7f8d72d9d640 1 -- 172.21.15.47:0/3993285149 reap_dead start 2024-09-15T21:56:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.137+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 shutdown_connections 2024-09-15T21:56:32.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.137+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 >> 172.21.15.47:0/3993285149 conn(0x7f8d7409e5f0 msgr2=0x7f8d7408ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.137+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 shutdown_connections 2024-09-15T21:56:32.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.137+0000 7f8d78a44640 1 -- 172.21.15.47:0/3993285149 wait complete. 2024-09-15T21:56:32.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.139+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 --> v1:172.21.15.47:6809/32770 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc61c12efa0 con 0x7fc600053810 2024-09-15T21:56:32.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.140+0000 7fc611ffb640 1 -- 172.21.15.47:0/1106466613 <== osd.3 v1:172.21.15.47:6809/32770 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1673232951) 0x7fc61c12efa0 con 0x7fc600053810 2024-09-15T21:56:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.140+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 >> v1:172.21.15.47:6809/32770 conn(0x7fc600053810 legacy=0x7fc600055c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.140+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 >> v1:172.21.15.47:6800/32267 conn(0x7fc60004d120 legacy=0x7fc60004f5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.140+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc61c135270 msgr2=0x7fc61c12e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.140+0000 7fc62335e640 1 --2- 172.21.15.47:0/1106466613 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc61c135270 0x7fc61c12e7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0953f0 tx=0x7fc61800d9c0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.140+0000 7fc6218fb640 1 -- 172.21.15.47:0/1106466613 reap_dead start 2024-09-15T21:56:32.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.140+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 shutdown_connections 2024-09-15T21:56:32.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.140+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 >> 172.21.15.47:0/1106466613 conn(0x7fc61c09c310 msgr2=0x7fc61c131b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.141+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 shutdown_connections 2024-09-15T21:56:32.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.141+0000 7fc62335e640 1 -- 172.21.15.47:0/1106466613 wait complete. 2024-09-15T21:56:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.145+0000 7f01a8880640 1 Processor -- start 2024-09-15T21:56:32.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.145+0000 7f01a8880640 1 -- start start 2024-09-15T21:56:32.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f01a8880640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 0x7f01a412a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.147 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f2e5472f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T21:56:32.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f01a8880640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01a412da80 0x7f01a41356b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f01a1d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f01a412ab10 0x7f01a412d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36794/0 (socket says 172.21.15.47:36794) 2024-09-15T21:56:32.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f01a2d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01a412da80 0x7f01a41356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f01a2d9d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01a412da80 0x7f01a41356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47548/0 (socket says 172.21.15.47:47548) 2024-09-15T21:56:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f01a1d9b640 1 -- 172.21.15.47:0/4095862790 learned_addr learned my addr 172.21.15.47:0/4095862790 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f01a8880640 1 -- 172.21.15.47:0/4095862790 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01a40995c0 con 0x7f01a412ab10 2024-09-15T21:56:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f01a8880640 1 -- 172.21.15.47:0/4095862790 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f01a409f9d0 con 0x7f01a412da80 2024-09-15T21:56:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f01a8880640 1 -- 172.21.15.47:0/4095862790 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f01a4099990 con 0x7f01a412a1f0 2024-09-15T21:56:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.146+0000 7f01a259c640 1 --2- 172.21.15.47:0/4095862790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 0x7f01a412a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.147+0000 7f01a159a640 1 -- 172.21.15.47:0/4095862790 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672037945 0 0) 0x7f0198001110 con 0x7f01a412ab10 2024-09-15T21:56:32.149 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:32.147+0000 7fd4a305c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T21:56:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.147+0000 7f01a159a640 1 -- 172.21.15.47:0/4095862790 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f018c003520 con 0x7f01a412ab10 2024-09-15T21:56:32.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.147+0000 7f01a2d9d640 1 -- 172.21.15.47:0/4095862790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 msgr2=0x7f01a412a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.147+0000 7f01a159a640 1 -- 172.21.15.47:0/4095862790 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1438615736 0 0) 0x7f018c003520 con 0x7f01a412ab10 2024-09-15T21:56:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.147+0000 7f01a2d9d640 1 --2- 172.21.15.47:0/4095862790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 0x7f01a412a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.147+0000 7f01a2d9d640 1 -- 172.21.15.47:0/4095862790 >> v1:172.21.15.47:6789/0 conn(0x7f01a412ab10 legacy=0x7f01a412d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.147+0000 7f01a2d9d640 1 -- 172.21.15.47:0/4095862790 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a4099ec0 con 0x7f01a412da80 2024-09-15T21:56:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.147+0000 7f01a259c640 1 --2- 172.21.15.47:0/4095862790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 0x7f01a412a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.148+0000 7f01a2d9d640 1 --2- 172.21.15.47:0/4095862790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a412da80 0x7f01a41356b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f019c009d90 tx=0x7f019c02a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28db4ce487bdd4c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.148+0000 7f01a159a640 1 -- 172.21.15.47:0/4095862790 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198001840 con 0x7f01a412da80 2024-09-15T21:56:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.148+0000 7f01a159a640 1 -- 172.21.15.47:0/4095862790 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019c007540 con 0x7f01a412da80 2024-09-15T21:56:32.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.148+0000 7f01a159a640 1 -- 172.21.15.47:0/4095862790 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c007910 con 0x7f01a412da80 2024-09-15T21:56:32.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.149+0000 7f39a7da3640 1 Processor -- start 2024-09-15T21:56:32.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.149+0000 7f39a7da3640 1 -- start start 2024-09-15T21:56:32.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.149+0000 7f01a8880640 1 -- 172.21.15.47:0/4095862790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a412da80 msgr2=0x7f01a41356b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.149+0000 7f01a8880640 1 --2- 172.21.15.47:0/4095862790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a412da80 0x7f01a41356b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f019c009d90 tx=0x7f019c02a550 comp rx=0 tx=0).stop 2024-09-15T21:56:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f01a8880640 1 -- 172.21.15.47:0/4095862790 shutdown_connections 2024-09-15T21:56:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f01a8880640 1 --2- 172.21.15.47:0/4095862790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a412da80 0x7f01a41356b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f01a8880640 1 --2- 172.21.15.47:0/4095862790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 0x7f01a412a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f01a8880640 1 -- 172.21.15.47:0/4095862790 >> 172.21.15.47:0/4095862790 conn(0x7f01a409db80 msgr2=0x7f01a409df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f39a7da3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39a0098070 0x7f39a01347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f39a7da3640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012de40 0x7f39a0136d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f39a7da3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a0071540 con 0x7f39a012d350 2024-09-15T21:56:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f39a7da3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f39a00712e0 con 0x7f39a0098070 2024-09-15T21:56:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f39a7da3640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f39a0138280 con 0x7f39a012de40 2024-09-15T21:56:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f39a5b3f640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f39a012d350 0x7f39a012d730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36810/0 (socket says 172.21.15.47:36810) 2024-09-15T21:56:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f39a5b3f640 1 -- 172.21.15.47:0/1926961762 learned_addr learned my addr 172.21.15.47:0/1926961762 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:32.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f39a6340640 1 --2- 172.21.15.47:0/1926961762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012de40 0x7f39a0136d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.150+0000 7f39a533e640 1 --2- 172.21.15.47:0/1926961762 >> v2:172.21.15.47:3300/0 conn(0x7f39a0098070 0x7f39a01347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.151+0000 7f01a8880640 1 -- 172.21.15.47:0/4095862790 shutdown_connections 2024-09-15T21:56:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.151+0000 7f39a4b3d640 1 -- 172.21.15.47:0/1926961762 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703982536 0 0) 0x7f39a0071540 con 0x7f39a012d350 2024-09-15T21:56:32.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.151+0000 7f39a4b3d640 1 -- 172.21.15.47:0/1926961762 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f398c003520 con 0x7f39a012d350 2024-09-15T21:56:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.151+0000 7f01a8880640 1 -- 172.21.15.47:0/4095862790 wait complete. 2024-09-15T21:56:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.151+0000 7f39a6340640 1 -- 172.21.15.47:0/1926961762 >> v2:172.21.15.47:3300/0 conn(0x7f39a0098070 msgr2=0x7f39a01347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.151+0000 7f39a6340640 1 --2- 172.21.15.47:0/1926961762 >> v2:172.21.15.47:3300/0 conn(0x7f39a0098070 0x7f39a01347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.151+0000 7f39a6340640 1 -- 172.21.15.47:0/1926961762 >> v1:172.21.15.47:6789/0 conn(0x7f39a012d350 legacy=0x7f39a012d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.151+0000 7f39a6340640 1 -- 172.21.15.47:0/1926961762 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a0138400 con 0x7f39a012de40 2024-09-15T21:56:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.152+0000 7f39a6340640 1 --2- 172.21.15.47:0/1926961762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012de40 0x7f39a0136d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3998009d90 tx=0x7f399802a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481b423d4ac7ad31 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.153+0000 7f39a4b3d640 1 -- 172.21.15.47:0/1926961762 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998007740 con 0x7f39a012de40 2024-09-15T21:56:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.153+0000 7f01a8880640 1 Processor -- start 2024-09-15T21:56:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.153+0000 7f39a4b3d640 1 -- 172.21.15.47:0/1926961762 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39980078e0 con 0x7f39a012de40 2024-09-15T21:56:32.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.153+0000 7f39a4b3d640 1 -- 172.21.15.47:0/1926961762 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998007cb0 con 0x7f39a012de40 2024-09-15T21:56:32.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.153+0000 7f01a8880640 1 -- start start 2024-09-15T21:56:32.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a8880640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 0x7f01a41bf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.158 INFO:teuthology.orchestra.run.smithi047.stdout:38654705683 2024-09-15T21:56:32.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a8880640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a412ab10 0x7f01a41bf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a259c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 0x7f01a41bf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f39a7da3640 1 -- 172.21.15.47:0/1926961762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012de40 msgr2=0x7f39a0136d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f39a7da3640 1 --2- 172.21.15.47:0/1926961762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012de40 0x7f39a0136d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3998009d90 tx=0x7f399802a530 comp rx=0 tx=0).stop 2024-09-15T21:56:32.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:56:32.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f39a7da3640 1 -- 172.21.15.47:0/1926961762 shutdown_connections 2024-09-15T21:56:32.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a412ab10 0x7f01a41bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a412ab10 0x7f01a41bf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47580/0 (socket says 172.21.15.47:47580) 2024-09-15T21:56:32.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a1d9b640 1 -- 172.21.15.47:0/1542385343 learned_addr learned my addr 172.21.15.47:0/1542385343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a8880640 1 --2- 172.21.15.47:0/1542385343 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f01a412da80 0x7f01a41bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01a412e030 con 0x7f01a412ab10 2024-09-15T21:56:32.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f01a412deb0 con 0x7f01a412a1f0 2024-09-15T21:56:32.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f01a412e1b0 con 0x7f01a412da80 2024-09-15T21:56:32.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f01a2d9d640 1 --2- 172.21.15.47:0/1542385343 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f01a412da80 0x7f01a41bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f39a7da3640 1 --2- 172.21.15.47:0/1926961762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012de40 0x7f39a0136d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f39a7da3640 1 --2- 172.21.15.47:0/1926961762 >> v2:172.21.15.47:3300/0 conn(0x7f39a0098070 0x7f39a01347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f39a7da3640 1 -- 172.21.15.47:0/1926961762 >> 172.21.15.47:0/1926961762 conn(0x7f39a009f020 msgr2=0x7f39a009f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.154+0000 7f39a7da3640 1 -- 172.21.15.47:0/1926961762 shutdown_connections 2024-09-15T21:56:32.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f01a1d9b640 1 -- 172.21.15.47:0/1542385343 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f01a412da80 msgr2=0x7f01a41bfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f01a1d9b640 1 --2- 172.21.15.47:0/1542385343 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f01a412da80 0x7f01a41bfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f01a1d9b640 1 -- 172.21.15.47:0/1542385343 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 msgr2=0x7f01a41bf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f01a1d9b640 1 --2- 172.21.15.47:0/1542385343 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 0x7f01a41bf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f01a1d9b640 1 -- 172.21.15.47:0/1542385343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a408f030 con 0x7f01a412ab10 2024-09-15T21:56:32.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f39a7da3640 1 -- 172.21.15.47:0/1926961762 wait complete. 2024-09-15T21:56:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f01a259c640 1 --2- 172.21.15.47:0/1542385343 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f01a412a1f0 0x7f01a41bf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f01a1d9b640 1 --2- 172.21.15.47:0/1542385343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a412ab10 0x7f01a41bf690 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f019800caf0 tx=0x7f019800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f0192ffd640 1 -- 172.21.15.47:0/1542385343 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198017070 con 0x7f01a412ab10 2024-09-15T21:56:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f0192ffd640 1 -- 172.21.15.47:0/1542385343 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0198005ad0 con 0x7f01a412ab10 2024-09-15T21:56:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a4129a20 con 0x7f01a412ab10 2024-09-15T21:56:32.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.155+0000 7f0192ffd640 1 -- 172.21.15.47:0/1542385343 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198007070 con 0x7f01a412ab10 2024-09-15T21:56:32.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.156+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a408e4e0 con 0x7f01a412ab10 2024-09-15T21:56:32.165 INFO:teuthology.orchestra.run.smithi047.stdout:38654705684 2024-09-15T21:56:32.166 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:56:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.156+0000 7f39a7da3640 1 Processor -- start 2024-09-15T21:56:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.157+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01a4075750 con 0x7f01a412ab10 2024-09-15T21:56:32.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.157+0000 7f39a7da3640 1 -- start start 2024-09-15T21:56:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.157+0000 7f0192ffd640 1 -- 172.21.15.47:0/1542385343 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7f019801a070 con 0x7f01a412ab10 2024-09-15T21:56:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.157+0000 7f01a259c640 1 -- 172.21.15.47:0/1542385343 >> v1:172.21.15.47:6800/32267 conn(0x7f018404d0a0 legacy=0x7f018404f560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.157+0000 7f0192ffd640 1 -- 172.21.15.47:0/1542385343 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7f0198062cd0 con 0x7f01a412ab10 2024-09-15T21:56:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.157+0000 7f39a7da3640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39a0098070 0x7f39a0124350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.157+0000 7f0192ffd640 1 -- 172.21.15.47:0/1542385343 --> v1:172.21.15.47:6801/32760 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f019801be50 con 0x7f0184053720 2024-09-15T21:56:32.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.157+0000 7f0192ffd640 1 -- 172.21.15.47:0/1542385343 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f0198065420 con 0x7f01a412ab10 2024-09-15T21:56:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.157+0000 7f39a7da3640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012d350 0x7f39a0124890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a5b3f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39a0098070 0x7f39a0124350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a5b3f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39a0098070 0x7f39a0124350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46322/0 (socket says 172.21.15.47:46322) 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a533e640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012d350 0x7f39a0124890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a5b3f640 1 -- 172.21.15.47:0/1121648526 learned_addr learned my addr 172.21.15.47:0/1121648526 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a7da3640 1 --2- 172.21.15.47:0/1121648526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a012de40 0x7f39a01c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f39a0137640 con 0x7f39a012de40 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f39a01374c0 con 0x7f39a012d350 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f39a01377c0 con 0x7f39a0098070 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a6340640 1 --2- 172.21.15.47:0/1121648526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a012de40 0x7f39a01c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a6340640 1 -- 172.21.15.47:0/1121648526 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39a0098070 msgr2=0x7f39a0124350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a6340640 1 --2- 172.21.15.47:0/1121648526 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39a0098070 0x7f39a0124350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a6340640 1 -- 172.21.15.47:0/1121648526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012d350 msgr2=0x7f39a0124890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a6340640 1 --2- 172.21.15.47:0/1121648526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012d350 0x7f39a0124890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a6340640 1 -- 172.21.15.47:0/1121648526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a01267e0 con 0x7f39a012de40 2024-09-15T21:56:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a533e640 1 --2- 172.21.15.47:0/1121648526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39a012d350 0x7f39a0124890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.158+0000 7f39a6340640 1 --2- 172.21.15.47:0/1121648526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a012de40 0x7f39a01c8710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f39a009c7a0 tx=0x7f3998004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.159+0000 7f0192ffd640 1 -- 172.21.15.47:0/1542385343 <== osd.0 v1:172.21.15.47:6801/32760 1 ==== command_reply(tid 1: 0 ) ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f019801be50 con 0x7f0184053720 2024-09-15T21:56:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.159+0000 7f39967fc640 1 -- 172.21.15.47:0/1121648526 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998040070 con 0x7f39a012de40 2024-09-15T21:56:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.159+0000 7f39967fc640 1 -- 172.21.15.47:0/1121648526 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3998031bc0 con 0x7f39a012de40 2024-09-15T21:56:32.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.159+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a0124dd0 con 0x7f39a012de40 2024-09-15T21:56:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.159+0000 7f39967fc640 1 -- 172.21.15.47:0/1121648526 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998009450 con 0x7f39a012de40 2024-09-15T21:56:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.159+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a0126cf0 con 0x7f39a012de40 2024-09-15T21:56:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.160+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39a01277a0 con 0x7f39a012de40 2024-09-15T21:56:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.160+0000 7f39967fc640 1 -- 172.21.15.47:0/1121648526 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7f3998039040 con 0x7f39a012de40 2024-09-15T21:56:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.161+0000 7f39967fc640 1 -- 172.21.15.47:0/1121648526 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7f3998084e70 con 0x7f39a012de40 2024-09-15T21:56:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.161+0000 7f39a5b3f640 1 -- 172.21.15.47:0/1121648526 >> v1:172.21.15.47:6800/32267 conn(0x7f398404d050 legacy=0x7f398404f510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.161+0000 7f39967fc640 1 -- 172.21.15.47:0/1121648526 --> v1:172.21.15.47:6813/32772 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3998006b90 con 0x7f3984053740 2024-09-15T21:56:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.161+0000 7f39967fc640 1 -- 172.21.15.47:0/1121648526 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f39980874d0 con 0x7f39a012de40 2024-09-15T21:56:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.162+0000 7f39967fc640 1 -- 172.21.15.47:0/1121648526 <== osd.2 v1:172.21.15.47:6813/32772 1 ==== command_reply(tid 1: 0 ) ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f3998006b90 con 0x7f3984053740 2024-09-15T21:56:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.186+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 --> v1:172.21.15.47:6801/32760 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f01a412ccb0 con 0x7f0184053720 2024-09-15T21:56:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.186+0000 7f0192ffd640 1 -- 172.21.15.47:0/1542385343 <== osd.0 v1:172.21.15.47:6801/32760 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 3077695196) 0x7f01a412ccb0 con 0x7f0184053720 2024-09-15T21:56:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.187+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 >> v1:172.21.15.47:6801/32760 conn(0x7f0184053720 legacy=0x7f0184055b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.188+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 >> v1:172.21.15.47:6800/32267 conn(0x7f018404d0a0 legacy=0x7f018404f560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.188+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a412ab10 msgr2=0x7f01a41bf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.188+0000 7f01a8880640 1 --2- 172.21.15.47:0/1542385343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01a412ab10 0x7f01a41bf690 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f019800caf0 tx=0x7f019800cef0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.188+0000 7f01a2d9d640 1 -- 172.21.15.47:0/1542385343 reap_dead start 2024-09-15T21:56:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.188+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 shutdown_connections 2024-09-15T21:56:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.188+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 >> 172.21.15.47:0/1542385343 conn(0x7f01a409db80 msgr2=0x7f01a412c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.188+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 shutdown_connections 2024-09-15T21:56:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.188+0000 7f01a8880640 1 -- 172.21.15.47:0/1542385343 wait complete. 2024-09-15T21:56:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.189+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 --> v1:172.21.15.47:6813/32772 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f39a012c190 con 0x7f3984053740 2024-09-15T21:56:32.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.190+0000 7f39967fc640 1 -- 172.21.15.47:0/1121648526 <== osd.2 v1:172.21.15.47:6813/32772 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1673232951) 0x7f39a012c190 con 0x7f3984053740 2024-09-15T21:56:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.190+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 >> v1:172.21.15.47:6813/32772 conn(0x7f3984053740 legacy=0x7f3984055b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.191+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 >> v1:172.21.15.47:6800/32267 conn(0x7f398404d050 legacy=0x7f398404f510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.191+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a012de40 msgr2=0x7f39a01c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.191+0000 7f39a7da3640 1 --2- 172.21.15.47:0/1121648526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39a012de40 0x7f39a01c8710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f39a009c7a0 tx=0x7f3998004a40 comp rx=0 tx=0).stop 2024-09-15T21:56:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.191+0000 7f39a6340640 1 -- 172.21.15.47:0/1121648526 reap_dead start 2024-09-15T21:56:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.191+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 shutdown_connections 2024-09-15T21:56:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.191+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 >> 172.21.15.47:0/1121648526 conn(0x7f39a009f020 msgr2=0x7f39a0072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.191+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 shutdown_connections 2024-09-15T21:56:32.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.191+0000 7f39a7da3640 1 -- 172.21.15.47:0/1121648526 wait complete. 2024-09-15T21:56:32.207 INFO:teuthology.orchestra.run.smithi047.stdout:38654705684 2024-09-15T21:56:32.207 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:56:32.208 INFO:teuthology.orchestra.run.smithi047.stdout:38654705683 2024-09-15T21:56:32.209 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:56:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.308+0000 7f117c908640 1 Processor -- start 2024-09-15T21:56:32.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.308+0000 7f117c908640 1 -- start start 2024-09-15T21:56:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f117c908640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1178129d90 0x7f117812a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f117c908640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f117812d9a0 0x7f117812de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f1175d9b640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f117812a6b0 0x7f117812d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36824/0 (socket says 172.21.15.47:36824) 2024-09-15T21:56:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f117659c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1178129d90 0x7f117812a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f1176d9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f117812d9a0 0x7f117812de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f117659c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1178129d90 0x7f117812a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47596/0 (socket says 172.21.15.47:47596) 2024-09-15T21:56:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f117c908640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1178099ef0 con 0x7f117812a6b0 2024-09-15T21:56:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f117c908640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1178099b20 con 0x7f1178129d90 2024-09-15T21:56:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f117c908640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f117812ab00 con 0x7f117812d9a0 2024-09-15T21:56:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f1175d9b640 1 -- 172.21.15.47:0/3777131009 learned_addr learned my addr 172.21.15.47:0/3777131009 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.309+0000 7f1176d9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f117812d9a0 0x7f117812de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39826/0 (socket says 172.21.15.47:39826) 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44723018 0 0) 0x7f1178099ef0 con 0x7f117812a6b0 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1160003520 con 0x7f117812a6b0 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1663759984 0 0) 0x7f1160003520 con 0x7f117812a6b0 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f117812d9a0 msgr2=0x7f117812de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 --2- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f117812d9a0 0x7f117812de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 >> v2:172.21.15.47:3300/0 conn(0x7f1178129d90 msgr2=0x7f117812a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 --2- 172.21.15.47:0/3777131009 >> v2:172.21.15.47:3300/0 conn(0x7f1178129d90 0x7f117812a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1178099750 con 0x7f117812a6b0 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2478300478 0 0) 0x7f116c001670 con 0x7f117812a6b0 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 >> v1:172.21.15.47:6789/0 conn(0x7f117812a6b0 legacy=0x7f117812d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 --2- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1160003a20 0x7f1160003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 --2- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1160004800 0x7f1160004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 --2- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11600051c0 0x7f116000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1178099b20 con 0x7f1160004800 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f117812ab00 con 0x7f11600051c0 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.310+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f116000bf00 con 0x7f1160003a20 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.311+0000 7f1175d9b640 1 --2- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1160004800 0x7f1160004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.311+0000 7f1175d9b640 1 -- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1160003a20 msgr2=0x7f1160003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.311+0000 7f1175d9b640 1 --2- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1160003a20 0x7f1160003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.311+0000 7f1175d9b640 1 -- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11600051c0 msgr2=0x7f116000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.311+0000 7f1175d9b640 1 --2- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f11600051c0 0x7f116000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.311+0000 7f1175d9b640 1 -- 172.21.15.47:0/3777131009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1160003520 con 0x7f1160004800 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.312+0000 7f1175d9b640 1 --2- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1160004800 0x7f1160004c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f116c00c630 tx=0x7f1160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38aadedb516f7b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.312+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c032020 con 0x7f1160004800 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.312+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116c005ad0 con 0x7f1160004800 2024-09-15T21:56:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.312+0000 7f117559a640 1 -- 172.21.15.47:0/3777131009 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c007d50 con 0x7f1160004800 2024-09-15T21:56:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.313+0000 7f117c908640 1 -- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1160004800 msgr2=0x7f1160004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.313+0000 7f117c908640 1 --2- 172.21.15.47:0/3777131009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1160004800 0x7f1160004c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f116c00c630 tx=0x7f1160005830 comp rx=0 tx=0).stop 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.313+0000 7f1176d9d640 1 -- 172.21.15.47:0/3777131009 reap_dead start 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.313+0000 7f117c908640 1 -- 172.21.15.47:0/3777131009 shutdown_connections 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.313+0000 7f117c908640 1 -- 172.21.15.47:0/3777131009 >> 172.21.15.47:0/3777131009 conn(0x7f117809fdf0 msgr2=0x7f11780a0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.313+0000 7f117c908640 1 -- 172.21.15.47:0/3777131009 shutdown_connections 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.313+0000 7f117c908640 1 -- 172.21.15.47:0/3777131009 wait complete. 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.315+0000 7f117c908640 1 Processor -- start 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.315+0000 7f117c908640 1 -- start start 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.315+0000 7f117c908640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1160004800 0x7f11781c03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.316+0000 7f117659c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1160004800 0x7f11781c03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.316+0000 7f117659c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1160004800 0x7f11781c03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39844/0 (socket says 172.21.15.47:39844) 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.316+0000 7f117659c640 1 -- 172.21.15.47:0/3782749463 learned_addr learned my addr 172.21.15.47:0/3782749463 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.316+0000 7f117c908640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11781c0910 0x7f1178070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.316+0000 7f1175d9b640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11781c0910 0x7f1178070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.316+0000 7f117c908640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11781c0d50 0x7f11780713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.317+0000 7f1176d9d640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11781c0d50 0x7f11780713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.317+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f117812e1a0 con 0x7f11781c0d50 2024-09-15T21:56:32.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.317+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f117812e020 con 0x7f1160004800 2024-09-15T21:56:32.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.317+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f117806de40 con 0x7f11781c0910 2024-09-15T21:56:32.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.318+0000 7f1175d9b640 1 -- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1160004800 msgr2=0x7f11781c03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.318+0000 7f1175d9b640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1160004800 0x7f11781c03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.318+0000 7f1175d9b640 1 -- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11781c0d50 msgr2=0x7f11780713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.318+0000 7f1175d9b640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11781c0d50 0x7f11780713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.318+0000 7f1175d9b640 1 -- 172.21.15.47:0/3782749463 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11780735b0 con 0x7f11781c0910 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.319+0000 7f1176d9d640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11781c0d50 0x7f11780713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.319+0000 7f117659c640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1160004800 0x7f11781c03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.319+0000 7f1175d9b640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11781c0910 0x7f1178070ea0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f116c030000 tx=0x7f1178099de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.319+0000 7f1166ffd640 1 -- 172.21.15.47:0/3782749463 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c032070 con 0x7f11781c0910 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.319+0000 7f1166ffd640 1 -- 172.21.15.47:0/3782749463 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116c007290 con 0x7f11781c0910 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.319+0000 7f1166ffd640 1 -- 172.21.15.47:0/3782749463 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116c007680 con 0x7f11781c0910 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.320+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1178074100 con 0x7f11781c0910 2024-09-15T21:56:32.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.320+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f1178071920 con 0x7f11781c0910 2024-09-15T21:56:32.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.322+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117812a170 con 0x7f11781c0910 2024-09-15T21:56:32.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.322+0000 7fac073a5640 1 Processor -- start 2024-09-15T21:56:32.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.322+0000 7f1166ffd640 1 -- 172.21.15.47:0/3782749463 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7f116c035020 con 0x7f11781c0910 2024-09-15T21:56:32.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.322+0000 7fac073a5640 1 -- start start 2024-09-15T21:56:32.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.323+0000 7fac073a5640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fac00136af0 0x7fac00138ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.323+0000 7fac073a5640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac00139420 0x7fac0013b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.323+0000 7fac073a5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac00095a60 con 0x7fac00133b40 2024-09-15T21:56:32.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.323+0000 7fac073a5640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fac00133780 con 0x7fac00136af0 2024-09-15T21:56:32.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.323+0000 7fac073a5640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fac00098240 con 0x7fac00139420 2024-09-15T21:56:32.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.323+0000 7f1166ffd640 1 -- 172.21.15.47:0/3782749463 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7f116c083580 con 0x7f11781c0910 2024-09-15T21:56:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.323+0000 7fac05141640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fac00133b40 0x7fac00133f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36840/0 (socket says 172.21.15.47:36840) 2024-09-15T21:56:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.323+0000 7fac05141640 1 -- 172.21.15.47:0/1276893804 learned_addr learned my addr 172.21.15.47:0/1276893804 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fac05942640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac00139420 0x7fac0013b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010428107 0 0) 0x7fac00095a60 con 0x7fac00133b40 2024-09-15T21:56:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabec003520 con 0x7fac00133b40 2024-09-15T21:56:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 747550291 0 0) 0x7fabec003520 con 0x7fac00133b40 2024-09-15T21:56:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac00139420 msgr2=0x7fac0013b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fac00139420 0x7fac0013b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fac00136af0 msgr2=0x7fac00138ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fac00136af0 0x7fac00138ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac000990b0 con 0x7fac00133b40 2024-09-15T21:56:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2478300478 0 0) 0x7fabfc0019f0 con 0x7fac00133b40 2024-09-15T21:56:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 >> v1:172.21.15.47:6789/0 conn(0x7fac00133b40 legacy=0x7fac00133f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fabec003a20 0x7fabec003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fabec004800 0x7fabec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fabec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.325+0000 7f117659c640 1 -- 172.21.15.47:0/3782749463 >> v1:172.21.15.47:6800/32267 conn(0x7f115804d050 legacy=0x7f115804f510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fabec00c160 con 0x7fabec005230 2024-09-15T21:56:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fabec00c310 con 0x7fabec004800 2024-09-15T21:56:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.324+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fabec00c510 con 0x7fabec003a20 2024-09-15T21:56:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.325+0000 7fac05942640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fabec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.325+0000 7fac05141640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fabec003a20 0x7fabec003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.325+0000 7fac05942640 1 -- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fabec003a20 msgr2=0x7fabec003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.325+0000 7fac05942640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fabec003a20 0x7fabec003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.325+0000 7fac05942640 1 -- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fabec004800 msgr2=0x7fabec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.325+0000 7fac05942640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fabec004800 0x7fabec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.325+0000 7fac05942640 1 -- 172.21.15.47:0/1276893804 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac000990b0 con 0x7fabec005230 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.326+0000 7fac05942640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fabec00b9c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac0009ca90 tx=0x7fabec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69771b7a6fa7d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.326+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8030070 con 0x7fabec005230 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.326+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf802ac60 con 0x7fabec005230 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.326+0000 7fabf7fff640 1 -- 172.21.15.47:0/1276893804 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf800b530 con 0x7fabec005230 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.327+0000 7fac073a5640 1 -- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 msgr2=0x7fabec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.327+0000 7fac073a5640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fabec00b9c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac0009ca90 tx=0x7fabec005830 comp rx=0 tx=0).stop 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.327+0000 7fac073a5640 1 -- 172.21.15.47:0/1276893804 shutdown_connections 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.327+0000 7fac073a5640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fabec00b9c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.327+0000 7fac073a5640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fabec004800 0x7fabec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.327+0000 7fac073a5640 1 --2- 172.21.15.47:0/1276893804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fabec003a20 0x7fabec003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.327+0000 7fac073a5640 1 -- 172.21.15.47:0/1276893804 >> 172.21.15.47:0/1276893804 conn(0x7fac0009f020 msgr2=0x7fac0009f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.327+0000 7fac073a5640 1 -- 172.21.15.47:0/1276893804 shutdown_connections 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.327+0000 7fac073a5640 1 -- 172.21.15.47:0/1276893804 wait complete. 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.328+0000 7fac073a5640 1 Processor -- start 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.328+0000 7fac073a5640 1 -- start start 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac073a5640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fabec003a20 0x7fac001c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac073a5640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fabec004800 0x7fac001c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac073a5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fac0006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac073a5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fac00136ed0 con 0x7fabec005230 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac073a5640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fac000978c0 con 0x7fabec003a20 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac073a5640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fac00137050 con 0x7fabec004800 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac05141640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fabec003a20 0x7fac001c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac05141640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fabec003a20 0x7fac001c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39876/0 (socket says 172.21.15.47:39876) 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac05942640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fac0006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac05942640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fac0006dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47644/0 (socket says 172.21.15.47:47644) 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac05141640 1 -- 172.21.15.47:0/1073359124 learned_addr learned my addr 172.21.15.47:0/1073359124 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.329+0000 7fac04940640 1 --2- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fabec004800 0x7fac001c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fac05942640 1 -- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fabec004800 msgr2=0x7fac001c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fac05942640 1 --2- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fabec004800 0x7fac001c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fac05942640 1 -- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fabec003a20 msgr2=0x7fac001c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fac05942640 1 --2- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fabec003a20 0x7fac001c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fac05942640 1 -- 172.21.15.47:0/1073359124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0006f800 con 0x7fabec005230 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fac05942640 1 --2- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fac0006dcd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fabec00c2e0 tx=0x7fabf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fabf5ffb640 1 -- 172.21.15.47:0/1073359124 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8043020 con 0x7fabec005230 2024-09-15T21:56:32.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fabf5ffb640 1 -- 172.21.15.47:0/1073359124 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf803f680 con 0x7fabec005230 2024-09-15T21:56:32.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fabf5ffb640 1 -- 172.21.15.47:0/1073359124 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf800b530 con 0x7fabec005230 2024-09-15T21:56:32.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fac073a5640 1 -- 172.21.15.47:0/1073359124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac000710c0 con 0x7fabec005230 2024-09-15T21:56:32.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.330+0000 7fac073a5640 1 -- 172.21.15.47:0/1073359124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac001c92a0 con 0x7fabec005230 2024-09-15T21:56:32.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.331+0000 7f1166ffd640 1 -- 172.21.15.47:0/3782749463 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7f116c05e730 con 0x7f11781c0910 2024-09-15T21:56:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.332+0000 7fac073a5640 1 -- 172.21.15.47:0/1073359124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac00138ae0 con 0x7fabec005230 2024-09-15T21:56:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.332+0000 7fabf5ffb640 1 -- 172.21.15.47:0/1073359124 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7fabf8046020 con 0x7fabec005230 2024-09-15T21:56:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.332+0000 7fabf5ffb640 1 -- 172.21.15.47:0/1073359124 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7fabf8084390 con 0x7fabec005230 2024-09-15T21:56:32.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.332+0000 7fac05141640 1 -- 172.21.15.47:0/1073359124 >> v1:172.21.15.47:6800/32267 conn(0x7fabe404d050 legacy=0x7fabe404f510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.339+0000 7fabf5ffb640 1 -- 172.21.15.47:0/1073359124 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7fac00138ae0 con 0x7fabec005230 2024-09-15T21:56:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.416+0000 7f3c47acb640 1 Processor -- start 2024-09-15T21:56:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.417+0000 7f3c47acb640 1 -- start start 2024-09-15T21:56:32.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.417+0000 7f9ae3824640 1 Processor -- start 2024-09-15T21:56:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.417+0000 7f3c47acb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3c4012e0c0 0x7f3c401328f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.417+0000 7f9ae3824640 1 -- start start 2024-09-15T21:56:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.417+0000 7f3c47acb640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c40132e30 0x7f3c40135220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.417+0000 7f3c45066640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3c4012e0c0 0x7f3c401328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f3c45066640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3c4012e0c0 0x7f3c401328f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47658/0 (socket says 172.21.15.47:47658) 2024-09-15T21:56:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f3c45066640 1 -- 172.21.15.47:0/2081860169 learned_addr learned my addr 172.21.15.47:0/2081860169 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f3c47acb640 1 -- 172.21.15.47:0/2081860169 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c40099260 con 0x7f3c4012d5d0 2024-09-15T21:56:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f3c47acb640 1 -- 172.21.15.47:0/2081860169 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3c4009a3a0 con 0x7f3c4012e0c0 2024-09-15T21:56:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f3c46068640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c40132e30 0x7f3c40135220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f3c47acb640 1 -- 172.21.15.47:0/2081860169 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f3c40135760 con 0x7f3c40132e30 2024-09-15T21:56:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f9ae3824640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc098040 0x7f9adc0963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f9ae3824640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096920 0x7f9adc12c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f9ae3824640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9adc12f4b0 con 0x7f9adc096d40 2024-09-15T21:56:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f9ae3824640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f9adc098ef0 con 0x7f9adc098040 2024-09-15T21:56:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f9ae3824640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f9adc06dd40 con 0x7f9adc096920 2024-09-15T21:56:32.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2677650359 0 0) 0x7f3c3c001110 con 0x7f3c4012d5d0 2024-09-15T21:56:32.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f9ae0dbf640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096920 0x7f9adc12c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f9ae0dbf640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096920 0x7f9adc12c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46390/0 (socket says 172.21.15.47:46390) 2024-09-15T21:56:32.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f9ae0dbf640 1 -- 172.21.15.47:0/2091837715 learned_addr learned my addr 172.21.15.47:0/2091837715 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c2c003520 con 0x7f3c4012d5d0 2024-09-15T21:56:32.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.418+0000 7f9ae15c0640 1 --2- 172.21.15.47:0/2091837715 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc098040 0x7f9adc0963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f9ae15c0640 1 -- 172.21.15.47:0/2091837715 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096920 msgr2=0x7f9adc12c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f9ae15c0640 1 --2- 172.21.15.47:0/2091837715 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096920 0x7f9adc12c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f9ae15c0640 1 -- 172.21.15.47:0/2091837715 >> v1:172.21.15.47:6789/0 conn(0x7f9adc096d40 legacy=0x7f9adc12c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f9ae15c0640 1 -- 172.21.15.47:0/2091837715 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9adc06e210 con 0x7f9adc098040 2024-09-15T21:56:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3418488947 0 0) 0x7f3c2c003520 con 0x7f3c4012d5d0 2024-09-15T21:56:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c40132e30 msgr2=0x7f3c40135220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c40132e30 0x7f3c40135220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 >> v2:172.21.15.47:3300/0 conn(0x7f3c4012e0c0 msgr2=0x7f3c401328f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 --2- 172.21.15.47:0/2081860169 >> v2:172.21.15.47:3300/0 conn(0x7f3c4012e0c0 0x7f3c401328f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c401358e0 con 0x7f3c4012d5d0 2024-09-15T21:56:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2478300478 0 0) 0x7f3c3c001840 con 0x7f3c4012d5d0 2024-09-15T21:56:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 >> v1:172.21.15.47:6789/0 conn(0x7f3c4012d5d0 legacy=0x7f3c4012d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2c003a50 0x7f3c2c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c40132e30 0x7f3c2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c2c004d40 0x7f3c2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4009a3a0 con 0x7f3c2c003a50 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f3c40135760 con 0x7f3c2c004d40 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.419+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f3c2c00bac0 con 0x7f3c40132e30 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f3c45066640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c40132e30 0x7f3c2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f3c45867640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2c003a50 0x7f3c2c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f3c46068640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c2c004d40 0x7f3c2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f3c45867640 1 -- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c40132e30 msgr2=0x7f3c2c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f3c45867640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c40132e30 0x7f3c2c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f3c45867640 1 -- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c2c004d40 msgr2=0x7f3c2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f3c45867640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c2c004d40 0x7f3c2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f3c45867640 1 -- 172.21.15.47:0/2081860169 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c401358e0 con 0x7f3c2c003a50 2024-09-15T21:56:32.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f3c45066640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c40132e30 0x7f3c2c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:32.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f9ae15c0640 1 --2- 172.21.15.47:0/2091837715 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc098040 0x7f9adc0963e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80097d0 tx=0x7f9ad802a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2629891049617ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.420+0000 7f3c46068640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c2c004d40 0x7f3c2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.421+0000 7f9ad3fff640 1 -- 172.21.15.47:0/2091837715 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8037070 con 0x7f9adc098040 2024-09-15T21:56:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.421+0000 7f9ad3fff640 1 -- 172.21.15.47:0/2091837715 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad802ad90 con 0x7f9adc098040 2024-09-15T21:56:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.421+0000 7f9ad3fff640 1 -- 172.21.15.47:0/2091837715 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8037070 con 0x7f9adc098040 2024-09-15T21:56:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.421+0000 7f9ae3824640 1 -- 172.21.15.47:0/2091837715 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc098040 msgr2=0x7f9adc0963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.421+0000 7f9ae3824640 1 --2- 172.21.15.47:0/2091837715 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc098040 0x7f9adc0963e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80097d0 tx=0x7f9ad802a040 comp rx=0 tx=0).stop 2024-09-15T21:56:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.422+0000 7f9ae3824640 1 -- 172.21.15.47:0/2091837715 shutdown_connections 2024-09-15T21:56:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.422+0000 7f9ae3824640 1 --2- 172.21.15.47:0/2091837715 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096920 0x7f9adc12c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.422+0000 7f9ae3824640 1 --2- 172.21.15.47:0/2091837715 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc098040 0x7f9adc0963e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.422+0000 7f9ae3824640 1 -- 172.21.15.47:0/2091837715 >> 172.21.15.47:0/2091837715 conn(0x7f9adc09ee50 msgr2=0x7f9adc09f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.422+0000 7f3c45867640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2c003a50 0x7f3c2c003e70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00c320 tx=0x7f3c2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bf00c7689e0e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.422+0000 7f9ae3824640 1 -- 172.21.15.47:0/2091837715 shutdown_connections 2024-09-15T21:56:32.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.422+0000 7f9ae3824640 1 -- 172.21.15.47:0/2091837715 wait complete. 2024-09-15T21:56:32.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.422+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c056020 con 0x7f3c2c003a50 2024-09-15T21:56:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.422+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c3c052680 con 0x7f3c2c003a50 2024-09-15T21:56:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.422+0000 7f3c44865640 1 -- 172.21.15.47:0/2081860169 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c007e10 con 0x7f3c2c003a50 2024-09-15T21:56:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.423+0000 7f3c47acb640 1 -- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2c003a50 msgr2=0x7f3c2c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.423+0000 7f3c47acb640 1 --2- 172.21.15.47:0/2081860169 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c2c003a50 0x7f3c2c003e70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00c320 tx=0x7f3c2c005830 comp rx=0 tx=0).stop 2024-09-15T21:56:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.423+0000 7f3c46068640 1 -- 172.21.15.47:0/2081860169 reap_dead start 2024-09-15T21:56:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.423+0000 7f3c47acb640 1 -- 172.21.15.47:0/2081860169 shutdown_connections 2024-09-15T21:56:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.423+0000 7f3c47acb640 1 -- 172.21.15.47:0/2081860169 >> 172.21.15.47:0/2081860169 conn(0x7f3c400a02b0 msgr2=0x7f3c400a06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.423+0000 7f3c47acb640 1 -- 172.21.15.47:0/2081860169 shutdown_connections 2024-09-15T21:56:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.423+0000 7f3c47acb640 1 -- 172.21.15.47:0/2081860169 wait complete. 2024-09-15T21:56:32.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.423+0000 7f9ae3824640 1 Processor -- start 2024-09-15T21:56:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.423+0000 7f9ae3824640 1 -- start start 2024-09-15T21:56:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.424+0000 7f3c47acb640 1 Processor -- start 2024-09-15T21:56:32.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.424+0000 7f3c47acb640 1 -- start start 2024-09-15T21:56:32.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.424+0000 7f9ae3824640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc096920 0x7f9adc134440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.424+0000 7f9ae3824640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096d40 0x7f9adc134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.424+0000 7f9ae15c0640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc096920 0x7f9adc134440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.424+0000 7f9ae15c0640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc096920 0x7f9adc134440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39902/0 (socket says 172.21.15.47:39902) 2024-09-15T21:56:32.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c47acb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c40096680 0x7f3c40096a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c47acb640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c4006de00 0x7f3c4006e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c47acb640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c4006e740 0x7f3c4006ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c47acb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3c401335a0 con 0x7f3c40096680 2024-09-15T21:56:32.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c47acb640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f3c40133420 con 0x7f3c4006e740 2024-09-15T21:56:32.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c47acb640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f3c40133720 con 0x7f3c4006de00 2024-09-15T21:56:32.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c45066640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c4006de00 0x7f3c4006e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c45867640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c40096680 0x7f3c40096a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f9ae3824640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc098040 0x7f9adc134ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:32.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae3824640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1336d0 con 0x7f9adc098040 2024-09-15T21:56:32.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae3824640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f9adc12d040 con 0x7f9adc096920 2024-09-15T21:56:32.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae3824640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f9adc133850 con 0x7f9adc096d40 2024-09-15T21:56:32.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae15c0640 1 -- 172.21.15.47:0/715240563 learned_addr learned my addr 172.21.15.47:0/715240563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c45867640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c40096680 0x7f3c40096a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47680/0 (socket says 172.21.15.47:47680) 2024-09-15T21:56:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c45867640 1 -- 172.21.15.47:0/539105762 learned_addr learned my addr 172.21.15.47:0/539105762 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c46068640 1 --2- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c4006e740 0x7f3c4006ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c45066640 1 -- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c4006e740 msgr2=0x7f3c4006ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c45066640 1 --2- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c4006e740 0x7f3c4006ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c45066640 1 -- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c40096680 msgr2=0x7f3c40096a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c45066640 1 --2- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c40096680 0x7f3c40096a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.425+0000 7f3c45066640 1 -- 172.21.15.47:0/539105762 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c40127f80 con 0x7f3c4006de00 2024-09-15T21:56:32.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f3c45066640 1 --2- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c4006de00 0x7f3c4006e200 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c30011ca0 tx=0x7f3c3000f610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae1dc1640 1 --2- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc098040 0x7f9adc134ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae1dc1640 1 -- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096d40 msgr2=0x7f9adc134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae0dbf640 1 --2- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096d40 0x7f9adc134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:32.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae1dc1640 1 --2- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096d40 0x7f9adc134980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae1dc1640 1 -- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc096920 msgr2=0x7f9adc134440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae1dc1640 1 --2- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc096920 0x7f9adc134440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae1dc1640 1 -- 172.21.15.47:0/715240563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9adc073a90 con 0x7f9adc098040 2024-09-15T21:56:32.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae0dbf640 1 --2- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096d40 0x7f9adc134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:56:32.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ae1dc1640 1 --2- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc098040 0x7f9adc134ec0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c690 tx=0x7f9ad400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:32.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.426+0000 7f9ad1ffb640 1 -- 172.21.15.47:0/715240563 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4003b70 con 0x7f9adc098040 2024-09-15T21:56:32.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.427+0000 7f9ae3824640 1 -- 172.21.15.47:0/715240563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9adc136640 con 0x7f9adc098040 2024-09-15T21:56:32.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.427+0000 7f9ad1ffb640 1 -- 172.21.15.47:0/715240563 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad4003d10 con 0x7f9adc098040 2024-09-15T21:56:32.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.427+0000 7f9ad1ffb640 1 -- 172.21.15.47:0/715240563 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4005310 con 0x7f9adc098040 2024-09-15T21:56:32.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.427+0000 7f3c367fc640 1 -- 172.21.15.47:0/539105762 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30019070 con 0x7f3c4006de00 2024-09-15T21:56:32.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.427+0000 7f3c367fc640 1 -- 172.21.15.47:0/539105762 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c30011e50 con 0x7f3c4006de00 2024-09-15T21:56:32.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.428+0000 7f3c367fc640 1 -- 172.21.15.47:0/539105762 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30004270 con 0x7f3c4006de00 2024-09-15T21:56:32.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.428+0000 7f3c47acb640 1 -- 172.21.15.47:0/539105762 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c40071330 con 0x7f3c4006de00 2024-09-15T21:56:32.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.428+0000 7f3c47acb640 1 -- 172.21.15.47:0/539105762 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4006ff40 con 0x7f3c4006de00 2024-09-15T21:56:32.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.428+0000 7f9ae3824640 1 -- 172.21.15.47:0/715240563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9adc137300 con 0x7f9adc098040 2024-09-15T21:56:32.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.429+0000 7f9ad1ffb640 1 -- 172.21.15.47:0/715240563 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7f9ad40054b0 con 0x7f9adc098040 2024-09-15T21:56:32.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.429+0000 7f9ad1ffb640 1 -- 172.21.15.47:0/715240563 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7f9ad4068390 con 0x7f9adc098040 2024-09-15T21:56:32.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.429+0000 7f9ae15c0640 1 -- 172.21.15.47:0/715240563 >> v1:172.21.15.47:6800/32267 conn(0x7f9ac004d050 legacy=0x7f9ac004f510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.429+0000 7f3c47acb640 1 -- 172.21.15.47:0/539105762 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4012d9b0 con 0x7f3c4006de00 2024-09-15T21:56:32.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.429+0000 7f9ae3824640 1 -- 172.21.15.47:0/715240563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9adc0963e0 con 0x7f9adc098040 2024-09-15T21:56:32.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.430+0000 7f3c367fc640 1 -- 172.21.15.47:0/539105762 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 6) ==== 58829+0+0 (secure 0 0 0) 0x7f3c3001c070 con 0x7f3c4006de00 2024-09-15T21:56:32.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.431+0000 7f3c45867640 1 -- 172.21.15.47:0/539105762 >> v1:172.21.15.47:6800/32267 conn(0x7f3c2404d0a0 legacy=0x7f3c2404f560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.431+0000 7f3c367fc640 1 -- 172.21.15.47:0/539105762 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(18..18 src has 1..18) ==== 2808+0+0 (secure 0 0 0) 0x7f3c30063d50 con 0x7f3c4006de00 2024-09-15T21:56:32.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.434+0000 7f9ad1ffb640 1 -- 172.21.15.47:0/715240563 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7f9ad403de10 con 0x7f9adc098040 2024-09-15T21:56:32.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.438+0000 7f3c367fc640 1 -- 172.21.15.47:0/539105762 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7f3c3003ef60 con 0x7f3c4006de00 2024-09-15T21:56:32.492 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:32.491+0000 7f2e5472f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T21:56:32.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.526+0000 7f117659c640 1 -- 172.21.15.47:0/3782749463 >> v1:172.21.15.47:6800/32267 conn(0x7f115804d050 legacy=0x7f115804f510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.533+0000 7fac05141640 1 -- 172.21.15.47:0/1073359124 >> v1:172.21.15.47:6800/32267 conn(0x7fabe404d050 legacy=0x7fabe404f510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.544 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:32.543+0000 7fd4a305c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T21:56:32.549 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:32.548+0000 7f2e5472f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T21:56:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.549+0000 7fac073a5640 1 -- 172.21.15.47:0/1073359124 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fac00075150 con 0x7fabec005230 2024-09-15T21:56:32.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.549+0000 7fabf5ffb640 1 -- 172.21.15.47:0/1073359124 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fabf804d030 con 0x7fabec005230 2024-09-15T21:56:32.550 INFO:teuthology.orchestra.run.smithi047.stdout:38654705682 2024-09-15T21:56:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.551+0000 7fac073a5640 1 -- 172.21.15.47:0/1073359124 >> v1:172.21.15.47:6800/32267 conn(0x7fabe404d050 legacy=0x7fabe404f510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:32.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.551+0000 7fac073a5640 1 -- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 msgr2=0x7fac0006dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.551+0000 7fac073a5640 1 --2- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fac0006dcd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fabec00c2e0 tx=0x7fabf8004730 comp rx=0 tx=0).stop 2024-09-15T21:56:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.551+0000 7fac073a5640 1 -- 172.21.15.47:0/1073359124 shutdown_connections 2024-09-15T21:56:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.551+0000 7fac073a5640 1 --2- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabec005230 0x7fac0006dcd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.551+0000 7fac073a5640 1 --2- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fabec004800 0x7fac001c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.551+0000 7fac073a5640 1 --2- 172.21.15.47:0/1073359124 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fabec003a20 0x7fac001c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.551+0000 7fac073a5640 1 -- 172.21.15.47:0/1073359124 >> 172.21.15.47:0/1073359124 conn(0x7fac0009f020 msgr2=0x7fac00099600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.551+0000 7fac073a5640 1 -- 172.21.15.47:0/1073359124 shutdown_connections 2024-09-15T21:56:32.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.551+0000 7fac073a5640 1 -- 172.21.15.47:0/1073359124 wait complete. 2024-09-15T21:56:32.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705684 got 38654705682 for osd.3 2024-09-15T21:56:32.621 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:32.619+0000 7fd4a305c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T21:56:32.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.631+0000 7f9ae15c0640 1 -- 172.21.15.47:0/715240563 >> v1:172.21.15.47:6800/32267 conn(0x7f9ac004d050 legacy=0x7f9ac004f510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.632+0000 7f3c45867640 1 -- 172.21.15.47:0/539105762 >> v1:172.21.15.47:6800/32267 conn(0x7f3c2404d0a0 legacy=0x7f3c2404f560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.47:6800/32267 2024-09-15T21:56:32.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.634+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f11781283f0 con 0x7f11781c0910 2024-09-15T21:56:32.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.634+0000 7f1166ffd640 1 -- 172.21.15.47:0/3782749463 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f116c082710 con 0x7f11781c0910 2024-09-15T21:56:32.636 INFO:teuthology.orchestra.run.smithi047.stdout:38654705681 2024-09-15T21:56:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.637+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 >> v1:172.21.15.47:6800/32267 conn(0x7f115804d050 legacy=0x7f115804f510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.637+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11781c0910 msgr2=0x7f1178070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.637+0000 7f117c908640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11781c0910 0x7f1178070ea0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f116c030000 tx=0x7f1178099de0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.637+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 shutdown_connections 2024-09-15T21:56:32.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.637+0000 7f117c908640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f11781c0d50 0x7f11780713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.637+0000 7f117c908640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f11781c0910 0x7f1178070ea0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.637+0000 7f117c908640 1 --2- 172.21.15.47:0/3782749463 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1160004800 0x7f11781c03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.637+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 >> 172.21.15.47:0/3782749463 conn(0x7f117809fdf0 msgr2=0x7f117809b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.638+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 shutdown_connections 2024-09-15T21:56:32.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.638+0000 7f117c908640 1 -- 172.21.15.47:0/3782749463 wait complete. 2024-09-15T21:56:32.641 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:32.640+0000 7f2e5472f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T21:56:32.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.649+0000 7f9ae3824640 1 -- 172.21.15.47:0/715240563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9adc068d50 con 0x7f9adc098040 2024-09-15T21:56:32.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.650+0000 7f9ad1ffb640 1 -- 172.21.15.47:0/715240563 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9ad40057a0 con 0x7f9adc098040 2024-09-15T21:56:32.651 INFO:teuthology.orchestra.run.smithi047.stdout:38654705682 2024-09-15T21:56:32.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.1 2024-09-15T21:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.653+0000 7f9ae3824640 1 -- 172.21.15.47:0/715240563 >> v1:172.21.15.47:6800/32267 conn(0x7f9ac004d050 legacy=0x7f9ac004f510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.653+0000 7f9ae3824640 1 -- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc098040 msgr2=0x7f9adc134ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.653+0000 7f9ae3824640 1 --2- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc098040 0x7f9adc134ec0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c690 tx=0x7f9ad400cb50 comp rx=0 tx=0).stop 2024-09-15T21:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.653+0000 7f9ae3824640 1 -- 172.21.15.47:0/715240563 shutdown_connections 2024-09-15T21:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.653+0000 7f9ae3824640 1 --2- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9adc098040 0x7f9adc134ec0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.653+0000 7f9ae3824640 1 --2- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9adc096d40 0x7f9adc134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.653+0000 7f9ae3824640 1 --2- 172.21.15.47:0/715240563 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9adc096920 0x7f9adc134440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.653+0000 7f9ae3824640 1 -- 172.21.15.47:0/715240563 >> 172.21.15.47:0/715240563 conn(0x7f9adc09ee50 msgr2=0x7f9adc12c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.653+0000 7f9ae3824640 1 -- 172.21.15.47:0/715240563 shutdown_connections 2024-09-15T21:56:32.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.653+0000 7f9ae3824640 1 -- 172.21.15.47:0/715240563 wait complete. 2024-09-15T21:56:32.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705684 got 38654705682 for osd.2 2024-09-15T21:56:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.704+0000 7f3c47acb640 1 -- 172.21.15.47:0/539105762 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3c400751c0 con 0x7f3c4006de00 2024-09-15T21:56:32.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.705+0000 7f3c367fc640 1 -- 172.21.15.47:0/539105762 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3c30004410 con 0x7f3c4006de00 2024-09-15T21:56:32.706 INFO:teuthology.orchestra.run.smithi047.stdout:38654705681 2024-09-15T21:56:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.707+0000 7f3c47acb640 1 -- 172.21.15.47:0/539105762 >> v1:172.21.15.47:6800/32267 conn(0x7f3c2404d0a0 legacy=0x7f3c2404f560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:56:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.707+0000 7f3c47acb640 1 -- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c4006de00 msgr2=0x7f3c4006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.707+0000 7f3c47acb640 1 --2- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c4006de00 0x7f3c4006e200 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c30011ca0 tx=0x7f3c3000f610 comp rx=0 tx=0).stop 2024-09-15T21:56:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.707+0000 7f3c47acb640 1 -- 172.21.15.47:0/539105762 shutdown_connections 2024-09-15T21:56:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.707+0000 7f3c47acb640 1 --2- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3c4006e740 0x7f3c4006ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.707+0000 7f3c47acb640 1 --2- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3c4006de00 0x7f3c4006e200 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.707+0000 7f3c47acb640 1 --2- 172.21.15.47:0/539105762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3c40096680 0x7f3c40096a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:32.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.707+0000 7f3c47acb640 1 -- 172.21.15.47:0/539105762 >> 172.21.15.47:0/539105762 conn(0x7f3c400a02b0 msgr2=0x7f3c40126f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.707+0000 7f3c47acb640 1 -- 172.21.15.47:0/539105762 shutdown_connections 2024-09-15T21:56:32.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:32.707+0000 7f3c47acb640 1 -- 172.21.15.47:0/539105762 wait complete. 2024-09-15T21:56:32.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.0 2024-09-15T21:56:32.728 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:32.726+0000 7fd4a305c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T21:56:32.744 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:32.743+0000 7f2e5472f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:56:32.793 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:32.792+0000 7f2e5472f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T21:56:32.828 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:32.826+0000 7fd4a305c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:56:32.872 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:32.871+0000 7fd4a305c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T21:56:32.898 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:32.897+0000 7f2e5472f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T21:56:32.953 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:32.952+0000 7f2e5472f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T21:56:32.976 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:32.975+0000 7fd4a305c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T21:56:33.001 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:32.999+0000 7f2e5472f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T21:56:33.030 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:33.029+0000 7fd4a305c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T21:56:33.074 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:33.073+0000 7fd4a305c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T21:56:33.117 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:33.115+0000 7f2e5472f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T21:56:33.171 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:33.170+0000 7f2e5472f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T21:56:33.181 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:33.180+0000 7fd4a305c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T21:56:33.225 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:33.224+0000 7f2e5472f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T21:56:33.231 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:33.230+0000 7fd4a305c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T21:56:33.282 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:33.280+0000 7fd4a305c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T21:56:33.297 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:33.295+0000 7f2e5472f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T21:56:33.338 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:33.336+0000 7fd4a305c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T21:56:33.344 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:33.342+0000 7f2e5472f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T21:56:33.382 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:33.381+0000 7fd4a305c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T21:56:33.390 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:33.389+0000 7f2e5472f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T21:56:33.426 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:33.425+0000 7fd4a305c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T21:56:33.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:56:33.630 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-09-15T21:56:33.628+0000 7f2e5472f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T21:56:33.655 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:56:33.660 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:33.659+0000 7fd4a305c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T21:56:33.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:56:33.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.671+0000 7fb6fe819640 1 Processor -- start 2024-09-15T21:56:33.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.671+0000 7fb6fe819640 1 -- start start 2024-09-15T21:56:33.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6fe819640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097180 0x7fb6f8097560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6fe819640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6f8097aa0 0x7fb6f8097e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6fe819640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f8099650 con 0x7fb6f8133e10 2024-09-15T21:56:33.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6fe819640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f80752c0 con 0x7fb6f8097180 2024-09-15T21:56:33.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6fe819640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fb6f8099090 con 0x7fb6f8097aa0 2024-09-15T21:56:33.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6f7fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097180 0x7fb6f8097560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6f7fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097180 0x7fb6f8097560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39920/0 (socket says 172.21.15.47:39920) 2024-09-15T21:56:33.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6f7fff640 1 -- 172.21.15.47:0/1092068435 learned_addr learned my addr 172.21.15.47:0/1092068435 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:33.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6f77fe640 1 --2- 172.21.15.47:0/1092068435 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6f8097aa0 0x7fb6f8097e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6f7fff640 1 -- 172.21.15.47:0/1092068435 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6f8097aa0 msgr2=0x7fb6f8097e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:33.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6f7fff640 1 --2- 172.21.15.47:0/1092068435 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6f8097aa0 0x7fb6f8097e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6f7fff640 1 -- 172.21.15.47:0/1092068435 >> v1:172.21.15.47:6789/0 conn(0x7fb6f8133e10 legacy=0x7fb6f8136200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.672+0000 7fb6f7fff640 1 -- 172.21.15.47:0/1092068435 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f8098db0 con 0x7fb6f8097180 2024-09-15T21:56:33.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.673+0000 7fb6f7fff640 1 --2- 172.21.15.47:0/1092068435 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097180 0x7fb6f8097560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009900 tx=0x7fb6e802a170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5b9895a38ee6f61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:33.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.674+0000 7fb6f6ffd640 1 -- 172.21.15.47:0/1092068435 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8037070 con 0x7fb6f8097180 2024-09-15T21:56:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.674+0000 7fb6f6ffd640 1 -- 172.21.15.47:0/1092068435 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e8032440 con 0x7fb6f8097180 2024-09-15T21:56:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.674+0000 7fb6f6ffd640 1 -- 172.21.15.47:0/1092068435 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8032810 con 0x7fb6f8097180 2024-09-15T21:56:33.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.675+0000 7fb6fe819640 1 -- 172.21.15.47:0/1092068435 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097180 msgr2=0x7fb6f8097560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.675+0000 7fb6fe819640 1 --2- 172.21.15.47:0/1092068435 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097180 0x7fb6f8097560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009900 tx=0x7fb6e802a170 comp rx=0 tx=0).stop 2024-09-15T21:56:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.675+0000 7fb6fe819640 1 -- 172.21.15.47:0/1092068435 shutdown_connections 2024-09-15T21:56:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.675+0000 7fb6fe819640 1 --2- 172.21.15.47:0/1092068435 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6f8097aa0 0x7fb6f8097e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.675+0000 7fb6fe819640 1 --2- 172.21.15.47:0/1092068435 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097180 0x7fb6f8097560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.675+0000 7fb6fe819640 1 -- 172.21.15.47:0/1092068435 >> 172.21.15.47:0/1092068435 conn(0x7fb6f809db80 msgr2=0x7fb6f809df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.675+0000 7fb6fe819640 1 -- 172.21.15.47:0/1092068435 shutdown_connections 2024-09-15T21:56:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.675+0000 7fb6fe819640 1 -- 172.21.15.47:0/1092068435 wait complete. 2024-09-15T21:56:33.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.676+0000 7fb6fe819640 1 Processor -- start 2024-09-15T21:56:33.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.677+0000 7fb6fe819640 1 -- start start 2024-09-15T21:56:33.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.677+0000 7fb6fe819640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6f8097180 0x7fb6f8090860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.677+0000 7fb6fe819640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097aa0 0x7fb6f8090da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.677+0000 7fb6fe819640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6f8133e10 0x7fb6f8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.677+0000 7fb6fe819640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8137aa0 con 0x7fb6f8097180 2024-09-15T21:56:33.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.677+0000 7fb6fe819640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f8137920 con 0x7fb6f8097aa0 2024-09-15T21:56:33.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.677+0000 7fb6fe819640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fb6f8137c20 con 0x7fb6f8133e10 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.678+0000 7fb6f77fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097aa0 0x7fb6f8090da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.678+0000 7fb6f77fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097aa0 0x7fb6f8090da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:39948/0 (socket says 172.21.15.47:39948) 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.678+0000 7fb6f77fe640 1 -- 172.21.15.47:0/758321295 learned_addr learned my addr 172.21.15.47:0/758321295 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.678+0000 7fb6fcdb6640 1 --2- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6f8133e10 0x7fb6f8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.678+0000 7fb6f77fe640 1 -- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6f8133e10 msgr2=0x7fb6f8070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.678+0000 7fb6f77fe640 1 --2- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6f8133e10 0x7fb6f8070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.678+0000 7fb6f77fe640 1 -- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6f8097180 msgr2=0x7fb6f8090860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.678+0000 7fb6f77fe640 1 --2- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6f8097180 0x7fb6f8090860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.678+0000 7fb6f77fe640 1 -- 172.21.15.47:0/758321295 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f8072a80 con 0x7fb6f8097aa0 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.679+0000 7fb6f77fe640 1 --2- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097aa0 0x7fb6f8090da0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00d7d0 tx=0x7fb6ec00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:33.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.679+0000 7fb6f4ff9640 1 -- 172.21.15.47:0/758321295 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec004020 con 0x7fb6f8097aa0 2024-09-15T21:56:33.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.679+0000 7fb6fe819640 1 -- 172.21.15.47:0/758321295 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f8073480 con 0x7fb6f8097aa0 2024-09-15T21:56:33.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.679+0000 7fb6f4ff9640 1 -- 172.21.15.47:0/758321295 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0041e0 con 0x7fb6f8097aa0 2024-09-15T21:56:33.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.679+0000 7fb6fe819640 1 -- 172.21.15.47:0/758321295 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f8071040 con 0x7fb6f8097aa0 2024-09-15T21:56:33.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.679+0000 7fb6f4ff9640 1 -- 172.21.15.47:0/758321295 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0043d0 con 0x7fb6f8097aa0 2024-09-15T21:56:33.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.680+0000 7fb6fe819640 1 -- 172.21.15.47:0/758321295 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f8134460 con 0x7fb6f8097aa0 2024-09-15T21:56:33.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.681+0000 7fb6f4ff9640 1 -- 172.21.15.47:0/758321295 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 7) ==== 55745+0+0 (secure 0 0 0) 0x7fb6ec01b020 con 0x7fb6f8097aa0 2024-09-15T21:56:33.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.681+0000 7fb6f4ff9640 1 -- 172.21.15.47:0/758321295 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7fb6ec00f040 con 0x7fb6f8097aa0 2024-09-15T21:56:33.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.685+0000 7fb6f4ff9640 1 -- 172.21.15.47:0/758321295 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7fb6f8134460 con 0x7fb6f8097aa0 2024-09-15T21:56:33.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:56:33.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.773+0000 7fdc227ae640 1 Processor -- start 2024-09-15T21:56:33.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.773+0000 7fdc227ae640 1 -- start start 2024-09-15T21:56:33.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.773+0000 7fdc227ae640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc1c12cfd0 0x7fdc1c1315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.773+0000 7fdc227ae640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c131b30 0x7fdc1c133f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.773+0000 7fdc227ae640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc1c071220 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc1b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc1c12cfd0 0x7fdc1c1315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc1bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdc1c12c2d0 0x7fdc1c12c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36888/0 (socket says 172.21.15.47:36888) 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc1b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdc1c12cfd0 0x7fdc1c1315f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47746/0 (socket says 172.21.15.47:47746) 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc1b7fe640 1 -- 172.21.15.47:0/1713208744 learned_addr learned my addr 172.21.15.47:0/1713208744 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc227ae640 1 -- 172.21.15.47:0/1713208744 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdc1c070110 con 0x7fdc1c12cfd0 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc20d4b640 1 --2- 172.21.15.47:0/1713208744 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c131b30 0x7fdc1c133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc227ae640 1 -- 172.21.15.47:0/1713208744 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fdc1c0713f0 con 0x7fdc1c131b30 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc1affd640 1 -- 172.21.15.47:0/1713208744 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2316174877 0 0) 0x7fdc1c071220 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc1affd640 1 -- 172.21.15.47:0/1713208744 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc04003520 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc20d4b640 1 -- 172.21.15.47:0/1713208744 >> v2:172.21.15.47:3300/0 conn(0x7fdc1c12cfd0 msgr2=0x7fdc1c1315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:33.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc20d4b640 1 --2- 172.21.15.47:0/1713208744 >> v2:172.21.15.47:3300/0 conn(0x7fdc1c12cfd0 0x7fdc1c1315f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc20d4b640 1 -- 172.21.15.47:0/1713208744 >> v1:172.21.15.47:6789/0 conn(0x7fdc1c12c2d0 legacy=0x7fdc1c12c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.774+0000 7fdc20d4b640 1 -- 172.21.15.47:0/1713208744 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c135470 con 0x7fdc1c131b30 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.775+0000 7fdc20d4b640 1 --2- 172.21.15.47:0/1713208744 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c131b30 0x7fdc1c133f20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009e40 tx=0x7fdc1402ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e05cc8285564272 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.775+0000 7fdc1affd640 1 -- 172.21.15.47:0/1713208744 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14035070 con 0x7fdc1c131b30 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.775+0000 7fdc1affd640 1 -- 172.21.15.47:0/1713208744 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc14031610 con 0x7fdc1c131b30 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.775+0000 7fdc1affd640 1 -- 172.21.15.47:0/1713208744 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc140319c0 con 0x7fdc1c131b30 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.776+0000 7fdc227ae640 1 -- 172.21.15.47:0/1713208744 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c131b30 msgr2=0x7fdc1c133f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.776+0000 7fdc227ae640 1 --2- 172.21.15.47:0/1713208744 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c131b30 0x7fdc1c133f20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009e40 tx=0x7fdc1402ca00 comp rx=0 tx=0).stop 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.776+0000 7fdc227ae640 1 -- 172.21.15.47:0/1713208744 shutdown_connections 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.776+0000 7fdc227ae640 1 --2- 172.21.15.47:0/1713208744 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c131b30 0x7fdc1c133f20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.776+0000 7fdc227ae640 1 --2- 172.21.15.47:0/1713208744 >> v2:172.21.15.47:3300/0 conn(0x7fdc1c12cfd0 0x7fdc1c1315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.776+0000 7fdc227ae640 1 -- 172.21.15.47:0/1713208744 >> 172.21.15.47:0/1713208744 conn(0x7fdc1c09b6e0 msgr2=0x7fdc1c09baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:33.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.776+0000 7fdc227ae640 1 -- 172.21.15.47:0/1713208744 shutdown_connections 2024-09-15T21:56:33.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.776+0000 7fdc227ae640 1 -- 172.21.15.47:0/1713208744 wait complete. 2024-09-15T21:56:33.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.778+0000 7fdc227ae640 1 Processor -- start 2024-09-15T21:56:33.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.778+0000 7fdc227ae640 1 -- start start 2024-09-15T21:56:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc227ae640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c12c2d0 0x7fdc1c1c0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc227ae640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc1c131b30 0x7fdc1c1c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc227ae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc1c124580 0x7fdc1c12f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc227ae640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c134830 con 0x7fdc1c124580 2024-09-15T21:56:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc227ae640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fdc1c1346b0 con 0x7fdc1c131b30 2024-09-15T21:56:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc227ae640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fdc1c1349b0 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc1bfff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c12c2d0 0x7fdc1c1c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc1bfff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c12c2d0 0x7fdc1c1c0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46548/0 (socket says 172.21.15.47:46548) 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc1bfff640 1 -- 172.21.15.47:0/1520914379 learned_addr learned my addr 172.21.15.47:0/1520914379 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc1b7fe640 1 --2- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc1c131b30 0x7fdc1c1c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc20d4b640 1 --2- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc1c124580 0x7fdc1c12f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc1bfff640 1 -- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc1c131b30 msgr2=0x7fdc1c1c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc1bfff640 1 --2- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc1c131b30 0x7fdc1c1c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc1bfff640 1 -- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc1c124580 msgr2=0x7fdc1c12f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc1bfff640 1 --2- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc1c124580 0x7fdc1c12f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc1bfff640 1 -- 172.21.15.47:0/1520914379 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c1312e0 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.779+0000 7fdc1b7fe640 1 --2- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc1c131b30 0x7fdc1c1c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.780+0000 7fdc1bfff640 1 --2- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c12c2d0 0x7fdc1c1c0620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c500 tx=0x7fdc0c00c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.780+0000 7fdc18ff9640 1 -- 172.21.15.47:0/1520914379 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c018070 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.780+0000 7fdc227ae640 1 -- 172.21.15.47:0/1520914379 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c1c10a0 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.780+0000 7fdc18ff9640 1 -- 172.21.15.47:0/1520914379 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc0c007500 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.780+0000 7fdc18ff9640 1 -- 172.21.15.47:0/1520914379 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c0049e0 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.780+0000 7fdc227ae640 1 -- 172.21.15.47:0/1520914379 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c12ffc0 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.782+0000 7fdc18ff9640 1 -- 172.21.15.47:0/1520914379 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 7) ==== 55745+0+0 (secure 0 0 0) 0x7fdc0c01b070 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.782+0000 7fdc18ff9640 1 -- 172.21.15.47:0/1520914379 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7fdc0c05ca80 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.782+0000 7fdc227ae640 1 -- 172.21.15.47:0/1520914379 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1c0726d0 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.785+0000 7f5e62511640 1 Processor -- start 2024-09-15T21:56:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.785+0000 7f5e62511640 1 -- start start 2024-09-15T21:56:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.786+0000 7f5e62511640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e5c098360 0x7f5e5c092830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.786+0000 7f5e62511640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c092d70 0x7f5e5c093170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.786+0000 7f5e62511640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e5c094f20 con 0x7f5e5c130440 2024-09-15T21:56:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.786+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c092d70 0x7f5e5c093170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.786+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c092d70 0x7f5e5c093170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40002/0 (socket says 172.21.15.47:40002) 2024-09-15T21:56:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.786+0000 7f5e5b7fe640 1 -- 172.21.15.47:0/3239699705 learned_addr learned my addr 172.21.15.47:0/3239699705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:33.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.786+0000 7f5e62511640 1 -- 172.21.15.47:0/3239699705 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5e5c093d10 con 0x7f5e5c098360 2024-09-15T21:56:33.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.786+0000 7f5e62511640 1 -- 172.21.15.47:0/3239699705 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f5e5c0936b0 con 0x7f5e5c092d70 2024-09-15T21:56:33.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.786+0000 7f5e5bfff640 1 --2- 172.21.15.47:0/3239699705 >> v2:172.21.15.47:3300/0 conn(0x7f5e5c098360 0x7f5e5c092830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.787+0000 7f5e5bfff640 1 -- 172.21.15.47:0/3239699705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c092d70 msgr2=0x7f5e5c093170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:33.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.787+0000 7f5e5bfff640 1 --2- 172.21.15.47:0/3239699705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c092d70 0x7f5e5c093170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.787+0000 7fdc18ff9640 1 -- 172.21.15.47:0/1520914379 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7fdc1c0726d0 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.787+0000 7f5e5affd640 1 -- 172.21.15.47:0/3239699705 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2334338433 0 0) 0x7f5e54001490 con 0x7f5e5c130440 2024-09-15T21:56:33.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.787+0000 7f5e5bfff640 1 -- 172.21.15.47:0/3239699705 >> v1:172.21.15.47:6789/0 conn(0x7f5e5c130440 legacy=0x7f5e5c132830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.787+0000 7f5e5bfff640 1 -- 172.21.15.47:0/3239699705 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5c0951c0 con 0x7f5e5c098360 2024-09-15T21:56:33.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.787+0000 7f5e5b7fe640 1 --2- 172.21.15.47:0/3239699705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c092d70 0x7f5e5c093170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:33.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.788+0000 7f5e5bfff640 1 --2- 172.21.15.47:0/3239699705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e5c098360 0x7f5e5c092830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0097d0 tx=0x7f5e4c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c9dff8d7fa45c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:33.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.788+0000 7f5e5affd640 1 -- 172.21.15.47:0/3239699705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c037070 con 0x7f5e5c098360 2024-09-15T21:56:33.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.788+0000 7f5e5affd640 1 -- 172.21.15.47:0/3239699705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e4c02adb0 con 0x7f5e5c098360 2024-09-15T21:56:33.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.788+0000 7f5e5affd640 1 -- 172.21.15.47:0/3239699705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c032660 con 0x7f5e5c098360 2024-09-15T21:56:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.789+0000 7f5e62511640 1 -- 172.21.15.47:0/3239699705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e5c098360 msgr2=0x7f5e5c092830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.789+0000 7f5e62511640 1 --2- 172.21.15.47:0/3239699705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e5c098360 0x7f5e5c092830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0097d0 tx=0x7f5e4c02a0a0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.789+0000 7f5e62511640 1 -- 172.21.15.47:0/3239699705 shutdown_connections 2024-09-15T21:56:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.789+0000 7f5e62511640 1 --2- 172.21.15.47:0/3239699705 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c092d70 0x7f5e5c093170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.789+0000 7f5e62511640 1 --2- 172.21.15.47:0/3239699705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e5c098360 0x7f5e5c092830 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.789+0000 7f5e62511640 1 -- 172.21.15.47:0/3239699705 >> 172.21.15.47:0/3239699705 conn(0x7f5e5c09e6f0 msgr2=0x7f5e5c09eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.789+0000 7f5e62511640 1 -- 172.21.15.47:0/3239699705 shutdown_connections 2024-09-15T21:56:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.789+0000 7f5e62511640 1 -- 172.21.15.47:0/3239699705 wait complete. 2024-09-15T21:56:33.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.790+0000 7f5e62511640 1 Processor -- start 2024-09-15T21:56:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.790+0000 7f5e62511640 1 -- start start 2024-09-15T21:56:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.790+0000 7f5e62511640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e5c092d70 0x7f5e5c1253f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.790+0000 7f5e62511640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c098360 0x7f5e5c125930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.790+0000 7f5e62511640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e5c130440 0x7f5e5c125e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e62511640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c068320 con 0x7f5e5c092d70 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e62511640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f5e5c0681a0 con 0x7f5e5c098360 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e62511640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f5e5c0684a0 con 0x7f5e5c130440 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e60aae640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e5c130440 0x7f5e5c125e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c098360 0x7f5e5c125930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e60aae640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e5c130440 0x7f5e5c125e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46556/0 (socket says 172.21.15.47:46556) 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e60aae640 1 -- 172.21.15.47:0/214792511 learned_addr learned my addr 172.21.15.47:0/214792511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e5bfff640 1 --2- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e5c092d70 0x7f5e5c1253f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e60aae640 1 -- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c098360 msgr2=0x7f5e5c125930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e60aae640 1 --2- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c098360 0x7f5e5c125930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e60aae640 1 -- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e5c092d70 msgr2=0x7f5e5c1253f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e60aae640 1 --2- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e5c092d70 0x7f5e5c1253f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e60aae640 1 -- 172.21.15.47:0/214792511 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5c128860 con 0x7f5e5c130440 2024-09-15T21:56:33.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e5b7fe640 1 --2- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c098360 0x7f5e5c125930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e5bfff640 1 --2- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e5c092d70 0x7f5e5c1253f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.791+0000 7f5e60aae640 1 --2- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e5c130440 0x7f5e5c125e70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400c780 tx=0x7f5e5400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.792+0000 7f5e58ff9640 1 -- 172.21.15.47:0/214792511 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54003da0 con 0x7f5e5c130440 2024-09-15T21:56:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.792+0000 7f5e62511640 1 -- 172.21.15.47:0/214792511 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5c127fc0 con 0x7f5e5c130440 2024-09-15T21:56:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.792+0000 7f5e58ff9640 1 -- 172.21.15.47:0/214792511 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e5400ce30 con 0x7f5e5c130440 2024-09-15T21:56:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.792+0000 7f5e58ff9640 1 -- 172.21.15.47:0/214792511 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54005270 con 0x7f5e5c130440 2024-09-15T21:56:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.792+0000 7f5e62511640 1 -- 172.21.15.47:0/214792511 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5c1c4800 con 0x7f5e5c130440 2024-09-15T21:56:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.793+0000 7f5e62511640 1 -- 172.21.15.47:0/214792511 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e5c0958b0 con 0x7f5e5c130440 2024-09-15T21:56:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.793+0000 7f5e58ff9640 1 -- 172.21.15.47:0/214792511 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 7) ==== 55745+0+0 (secure 0 0 0) 0x7f5e54005410 con 0x7f5e5c130440 2024-09-15T21:56:33.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.794+0000 7f5e58ff9640 1 -- 172.21.15.47:0/214792511 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7f5e5405d280 con 0x7f5e5c130440 2024-09-15T21:56:33.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.797+0000 7f5e58ff9640 1 -- 172.21.15.47:0/214792511 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7f5e5c0958b0 con 0x7f5e5c130440 2024-09-15T21:56:33.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.832+0000 7fbf81a96640 1 Processor -- start 2024-09-15T21:56:33.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.832+0000 7fbf81a96640 1 -- start start 2024-09-15T21:56:33.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.832+0000 7fbf81a96640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf7c09a630 0x7fbf7c09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.832+0000 7fbf81a96640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbf7c09af50 0x7fbf7c12f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.832+0000 7fbf81a96640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7c09b600 con 0x7fbf7c12f920 2024-09-15T21:56:33.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.832+0000 7fbf81a96640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbf7c06d480 con 0x7fbf7c09a630 2024-09-15T21:56:33.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.832+0000 7fbf81a96640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fbf7c06d140 con 0x7fbf7c09af50 2024-09-15T21:56:33.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.832+0000 7fbf7b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf7c09a630 0x7fbf7c09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.833+0000 7fbf73fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbf7c09af50 0x7fbf7c12f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.833+0000 7fbf7bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbf7c12f920 0x7fbf7c12fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36914/0 (socket says 172.21.15.47:36914) 2024-09-15T21:56:33.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.833+0000 7fbf7b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf7c09a630 0x7fbf7c09aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47778/0 (socket says 172.21.15.47:47778) 2024-09-15T21:56:33.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.833+0000 7fbf7bfff640 1 -- 172.21.15.47:0/3488820904 learned_addr learned my addr 172.21.15.47:0/3488820904 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:33.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.833+0000 7fbf7b7fe640 1 -- 172.21.15.47:0/3488820904 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbf7c09af50 msgr2=0x7fbf7c12f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:33.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.833+0000 7fbf7b7fe640 1 --2- 172.21.15.47:0/3488820904 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbf7c09af50 0x7fbf7c12f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.833+0000 7fbf7b7fe640 1 -- 172.21.15.47:0/3488820904 >> v1:172.21.15.47:6789/0 conn(0x7fbf7c12f920 legacy=0x7fbf7c12fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.833+0000 7fbf7b7fe640 1 -- 172.21.15.47:0/3488820904 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c09bc00 con 0x7fbf7c09a630 2024-09-15T21:56:33.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.834+0000 7fbf7b7fe640 1 --2- 172.21.15.47:0/3488820904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c09a630 0x7fbf7c09aa10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0097a0 tx=0x7fbf6c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d610d6ea7f01c10 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:33.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.834+0000 7fbf7affd640 1 -- 172.21.15.47:0/3488820904 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c037070 con 0x7fbf7c09a630 2024-09-15T21:56:33.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.834+0000 7fbf7affd640 1 -- 172.21.15.47:0/3488820904 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6c02adb0 con 0x7fbf7c09a630 2024-09-15T21:56:33.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.834+0000 7fbf7affd640 1 -- 172.21.15.47:0/3488820904 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c032660 con 0x7fbf7c09a630 2024-09-15T21:56:33.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.835+0000 7fbf81a96640 1 -- 172.21.15.47:0/3488820904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c09a630 msgr2=0x7fbf7c09aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.835+0000 7fbf81a96640 1 --2- 172.21.15.47:0/3488820904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c09a630 0x7fbf7c09aa10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0097a0 tx=0x7fbf6c02a060 comp rx=0 tx=0).stop 2024-09-15T21:56:33.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.835+0000 7fbf81a96640 1 -- 172.21.15.47:0/3488820904 shutdown_connections 2024-09-15T21:56:33.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.835+0000 7fbf81a96640 1 --2- 172.21.15.47:0/3488820904 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbf7c09af50 0x7fbf7c12f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.835+0000 7fbf81a96640 1 --2- 172.21.15.47:0/3488820904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c09a630 0x7fbf7c09aa10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.835+0000 7fbf81a96640 1 -- 172.21.15.47:0/3488820904 >> 172.21.15.47:0/3488820904 conn(0x7fbf7c09eb90 msgr2=0x7fbf7c09efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:33.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.835+0000 7fbf81a96640 1 -- 172.21.15.47:0/3488820904 shutdown_connections 2024-09-15T21:56:33.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.835+0000 7fbf81a96640 1 -- 172.21.15.47:0/3488820904 wait complete. 2024-09-15T21:56:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf81a96640 1 Processor -- start 2024-09-15T21:56:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf81a96640 1 -- start start 2024-09-15T21:56:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf81a96640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c09a630 0x7fbf7c130f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf81a96640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbf7c09af50 0x7fbf7c1314a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf7b7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c09a630 0x7fbf7c130f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf81a96640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c12f920 0x7fbf7c1319e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf81a96640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c137ad0 con 0x7fbf7c12f920 2024-09-15T21:56:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf7b7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c09a630 0x7fbf7c130f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40022/0 (socket says 172.21.15.47:40022) 2024-09-15T21:56:33.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf73fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbf7c09af50 0x7fbf7c1314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf7b7fe640 1 -- 172.21.15.47:0/1262224199 learned_addr learned my addr 172.21.15.47:0/1262224199 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf81a96640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7c137950 con 0x7fbf7c09a630 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.837+0000 7fbf81a96640 1 -- 172.21.15.47:0/1262224199 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fbf7c137c50 con 0x7fbf7c09af50 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf7bfff640 1 --2- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c12f920 0x7fbf7c1319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf7b7fe640 1 -- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbf7c09af50 msgr2=0x7fbf7c1314a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf7b7fe640 1 --2- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbf7c09af50 0x7fbf7c1314a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf7b7fe640 1 -- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c12f920 msgr2=0x7fbf7c1319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf7b7fe640 1 --2- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c12f920 0x7fbf7c1319e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf7b7fe640 1 -- 172.21.15.47:0/1262224199 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c133970 con 0x7fbf7c09a630 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf7b7fe640 1 --2- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c09a630 0x7fbf7c130f60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0098c0 tx=0x7fbf6c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf78ff9640 1 -- 172.21.15.47:0/1262224199 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c041030 con 0x7fbf7c09a630 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf81a96640 1 -- 172.21.15.47:0/1262224199 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7c133500 con 0x7fbf7c09a630 2024-09-15T21:56:33.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf78ff9640 1 -- 172.21.15.47:0/1262224199 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6c03c670 con 0x7fbf7c09a630 2024-09-15T21:56:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.838+0000 7fbf81a96640 1 -- 172.21.15.47:0/1262224199 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7c131f20 con 0x7fbf7c09a630 2024-09-15T21:56:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.839+0000 7fbf78ff9640 1 -- 172.21.15.47:0/1262224199 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c037040 con 0x7fbf7c09a630 2024-09-15T21:56:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.840+0000 7fbf81a96640 1 -- 172.21.15.47:0/1262224199 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf60005290 con 0x7fbf7c09a630 2024-09-15T21:56:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.840+0000 7fbf78ff9640 1 -- 172.21.15.47:0/1262224199 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 7) ==== 55745+0+0 (secure 0 0 0) 0x7fbf6c044070 con 0x7fbf7c09a630 2024-09-15T21:56:33.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.840+0000 7fbf78ff9640 1 -- 172.21.15.47:0/1262224199 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7fbf6c07db50 con 0x7fbf7c09a630 2024-09-15T21:56:33.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.847+0000 7fbf78ff9640 1 -- 172.21.15.47:0/1262224199 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+142737 (secure 0 0 0) 0x7fbf7c133500 con 0x7fbf7c09a630 2024-09-15T21:56:33.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.849+0000 7fb6fe819640 1 -- 172.21.15.47:0/758321295 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb6f8075730 con 0x7fb6f8097aa0 2024-09-15T21:56:33.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.849+0000 7fb6f4ff9640 1 -- 172.21.15.47:0/758321295 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb6ec03c5a0 con 0x7fb6f8097aa0 2024-09-15T21:56:33.851 INFO:teuthology.orchestra.run.smithi047.stdout:38654705682 2024-09-15T21:56:33.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.851+0000 7fb6fe819640 1 -- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097aa0 msgr2=0x7fb6f8090da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.851+0000 7fb6fe819640 1 --2- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097aa0 0x7fb6f8090da0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00d7d0 tx=0x7fb6ec00dc90 comp rx=0 tx=0).stop 2024-09-15T21:56:33.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.851+0000 7fb6fe819640 1 -- 172.21.15.47:0/758321295 shutdown_connections 2024-09-15T21:56:33.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.851+0000 7fb6fe819640 1 --2- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb6f8133e10 0x7fb6f8070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.851+0000 7fb6fe819640 1 --2- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb6f8097aa0 0x7fb6f8090da0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.851+0000 7fb6fe819640 1 --2- 172.21.15.47:0/758321295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6f8097180 0x7fb6f8090860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.851+0000 7fb6fe819640 1 -- 172.21.15.47:0/758321295 >> 172.21.15.47:0/758321295 conn(0x7fb6f809db80 msgr2=0x7fb6f809b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:33.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.851+0000 7fb6fe819640 1 -- 172.21.15.47:0/758321295 shutdown_connections 2024-09-15T21:56:33.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.851+0000 7fb6fe819640 1 -- 172.21.15.47:0/758321295 wait complete. 2024-09-15T21:56:33.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705684 got 38654705682 for osd.3 2024-09-15T21:56:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.954+0000 7fdc227ae640 1 -- 172.21.15.47:0/1520914379 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdc1c134dc0 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.955+0000 7fdc18ff9640 1 -- 172.21.15.47:0/1520914379 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc0c03bee0 con 0x7fdc1c12c2d0 2024-09-15T21:56:33.956 INFO:teuthology.orchestra.run.smithi047.stdout:38654705681 2024-09-15T21:56:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.956+0000 7fdc227ae640 1 -- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c12c2d0 msgr2=0x7fdc1c1c0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.956+0000 7fdc227ae640 1 --2- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c12c2d0 0x7fdc1c1c0620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c500 tx=0x7fdc0c00c9c0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.956+0000 7fdc227ae640 1 -- 172.21.15.47:0/1520914379 shutdown_connections 2024-09-15T21:56:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.956+0000 7fdc227ae640 1 --2- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc1c124580 0x7fdc1c12f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.956+0000 7fdc227ae640 1 --2- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdc1c131b30 0x7fdc1c1c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.956+0000 7fdc227ae640 1 --2- 172.21.15.47:0/1520914379 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdc1c12c2d0 0x7fdc1c1c0620 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.956+0000 7fdc227ae640 1 -- 172.21.15.47:0/1520914379 >> 172.21.15.47:0/1520914379 conn(0x7fdc1c09b6e0 msgr2=0x7fdc1c12a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.956+0000 7fdc227ae640 1 -- 172.21.15.47:0/1520914379 shutdown_connections 2024-09-15T21:56:33.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.956+0000 7fdc227ae640 1 -- 172.21.15.47:0/1520914379 wait complete. 2024-09-15T21:56:33.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.960+0000 7f5e62511640 1 -- 172.21.15.47:0/214792511 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5e5c0687c0 con 0x7f5e5c130440 2024-09-15T21:56:33.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.960+0000 7f5e58ff9640 1 -- 172.21.15.47:0/214792511 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5e54020020 con 0x7f5e5c130440 2024-09-15T21:56:33.962 INFO:teuthology.orchestra.run.smithi047.stdout:38654705682 2024-09-15T21:56:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.962+0000 7f5e62511640 1 -- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e5c130440 msgr2=0x7f5e5c125e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.962+0000 7f5e62511640 1 --2- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e5c130440 0x7f5e5c125e70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400c780 tx=0x7f5e5400cc40 comp rx=0 tx=0).stop 2024-09-15T21:56:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.962+0000 7f5e62511640 1 -- 172.21.15.47:0/214792511 shutdown_connections 2024-09-15T21:56:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.962+0000 7f5e62511640 1 --2- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5e5c130440 0x7f5e5c125e70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.962+0000 7f5e62511640 1 --2- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5e5c098360 0x7f5e5c125930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.962+0000 7f5e62511640 1 --2- 172.21.15.47:0/214792511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e5c092d70 0x7f5e5c1253f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.962+0000 7f5e62511640 1 -- 172.21.15.47:0/214792511 >> 172.21.15.47:0/214792511 conn(0x7f5e5c09e6f0 msgr2=0x7f5e5c095f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:33.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.962+0000 7f5e62511640 1 -- 172.21.15.47:0/214792511 shutdown_connections 2024-09-15T21:56:33.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.962+0000 7f5e62511640 1 -- 172.21.15.47:0/214792511 wait complete. 2024-09-15T21:56:33.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.1 2024-09-15T21:56:33.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705684 got 38654705682 for osd.2 2024-09-15T21:56:33.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.989+0000 7fbf81a96640 1 -- 172.21.15.47:0/1262224199 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fbf60005580 con 0x7fbf7c09a630 2024-09-15T21:56:33.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.989+0000 7fbf78ff9640 1 -- 172.21.15.47:0/1262224199 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbf7c133970 con 0x7fbf7c09a630 2024-09-15T21:56:33.991 INFO:teuthology.orchestra.run.smithi047.stdout:38654705681 2024-09-15T21:56:33.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.991+0000 7fbf81a96640 1 -- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c09a630 msgr2=0x7fbf7c130f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:33.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.991+0000 7fbf81a96640 1 --2- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c09a630 0x7fbf7c130f60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0098c0 tx=0x7fbf6c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.991+0000 7fbf81a96640 1 -- 172.21.15.47:0/1262224199 shutdown_connections 2024-09-15T21:56:33.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.991+0000 7fbf81a96640 1 --2- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf7c12f920 0x7fbf7c1319e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.991+0000 7fbf81a96640 1 --2- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fbf7c09af50 0x7fbf7c1314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.991+0000 7fbf81a96640 1 --2- 172.21.15.47:0/1262224199 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fbf7c09a630 0x7fbf7c130f60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:33.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.991+0000 7fbf81a96640 1 -- 172.21.15.47:0/1262224199 >> 172.21.15.47:0/1262224199 conn(0x7fbf7c09eb90 msgr2=0x7fbf7c06edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:33.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.991+0000 7fbf81a96640 1 -- 172.21.15.47:0/1262224199 shutdown_connections 2024-09-15T21:56:33.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:33.991+0000 7fbf81a96640 1 -- 172.21.15.47:0/1262224199 wait complete. 2024-09-15T21:56:34.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.0 2024-09-15T21:56:34.674 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:34.673+0000 7fd48c872640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-15T21:56:34.675 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:34.674+0000 7fd48c872640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T21:56:34.675 INFO:tasks.ceph.mgr.x.smithi047.stderr:2024-09-15T21:56:34.674+0000 7fd48c872640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T21:56:34.865 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:56:34.970 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:56:34.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:56:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51d1733640 1 Processor -- start 2024-09-15T21:56:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51d1733640 1 -- start start 2024-09-15T21:56:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51d1733640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09a630 0x7f51cc09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51d1733640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc12faf0 0x7f51cc12ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51d1733640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51cc09b600 con 0x7f51cc09af50 2024-09-15T21:56:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51d1733640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f51cc06d480 con 0x7f51cc09a630 2024-09-15T21:56:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51d1733640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f51cc06d140 con 0x7f51cc12faf0 2024-09-15T21:56:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51caffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09a630 0x7f51cc09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:34.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51cb7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc12faf0 0x7f51cc12ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51caffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09a630 0x7f51cc09aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40030/0 (socket says 172.21.15.47:40030) 2024-09-15T21:56:34.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.979+0000 7f51caffd640 1 -- 172.21.15.47:0/686503061 learned_addr learned my addr 172.21.15.47:0/686503061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.980+0000 7f51caffd640 1 -- 172.21.15.47:0/686503061 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc12faf0 msgr2=0x7f51cc12ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.980+0000 7f51caffd640 1 --2- 172.21.15.47:0/686503061 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc12faf0 0x7f51cc12ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.980+0000 7f51caffd640 1 -- 172.21.15.47:0/686503061 >> v1:172.21.15.47:6789/0 conn(0x7f51cc09af50 legacy=0x7f51cc12f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.980+0000 7f51caffd640 1 -- 172.21.15.47:0/686503061 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc09bc00 con 0x7f51cc09a630 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.980+0000 7f51cb7fe640 1 --2- 172.21.15.47:0/686503061 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc12faf0 0x7f51cc12ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.981+0000 7f51caffd640 1 --2- 172.21.15.47:0/686503061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09a630 0x7f51cc09aa10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0098a0 tx=0x7f51bc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec058cdb10b24af server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.981+0000 7f51c9ffb640 1 -- 172.21.15.47:0/686503061 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc037070 con 0x7f51cc09a630 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.981+0000 7f51c9ffb640 1 -- 172.21.15.47:0/686503061 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51bc032440 con 0x7f51cc09a630 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.981+0000 7f51c9ffb640 1 -- 172.21.15.47:0/686503061 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc02ae60 con 0x7f51cc09a630 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.982+0000 7f51d1733640 1 -- 172.21.15.47:0/686503061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09a630 msgr2=0x7f51cc09aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.982+0000 7f51d1733640 1 --2- 172.21.15.47:0/686503061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09a630 0x7f51cc09aa10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0098a0 tx=0x7f51bc02a110 comp rx=0 tx=0).stop 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.982+0000 7f51d1733640 1 -- 172.21.15.47:0/686503061 shutdown_connections 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.982+0000 7f51d1733640 1 --2- 172.21.15.47:0/686503061 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc12faf0 0x7f51cc12ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.982+0000 7f51d1733640 1 --2- 172.21.15.47:0/686503061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09a630 0x7f51cc09aa10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.982+0000 7f51d1733640 1 -- 172.21.15.47:0/686503061 >> 172.21.15.47:0/686503061 conn(0x7f51cc09eb90 msgr2=0x7f51cc09efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:34.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.982+0000 7f51d1733640 1 -- 172.21.15.47:0/686503061 shutdown_connections 2024-09-15T21:56:34.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.982+0000 7f51d1733640 1 -- 172.21.15.47:0/686503061 wait complete. 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.983+0000 7f51d1733640 1 Processor -- start 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.983+0000 7f51d1733640 1 -- start start 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51d1733640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc09a630 0x7f51cc131160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51d1733640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09af50 0x7f51cc1316a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51d1733640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51cc12faf0 0x7f51cc131be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51d1733640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc137ad0 con 0x7f51cc12faf0 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51d1733640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f51cc137950 con 0x7f51cc09af50 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51d1733640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f51cc137c50 con 0x7f51cc09a630 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51caffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc09a630 0x7f51cc131160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51ca7fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09af50 0x7f51cc1316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:34.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51cb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51cc12faf0 0x7f51cc131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51caffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc09a630 0x7f51cc131160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46606/0 (socket says 172.21.15.47:46606) 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51caffd640 1 -- 172.21.15.47:0/2182502753 learned_addr learned my addr 172.21.15.47:0/2182502753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51ca7fc640 1 -- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc09a630 msgr2=0x7f51cc131160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51ca7fc640 1 --2- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc09a630 0x7f51cc131160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51ca7fc640 1 -- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51cc12faf0 msgr2=0x7f51cc131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51ca7fc640 1 --2- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51cc12faf0 0x7f51cc131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51ca7fc640 1 -- 172.21.15.47:0/2182502753 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc132a60 con 0x7f51cc09af50 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51cb7fe640 1 --2- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51cc12faf0 0x7f51cc131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.984+0000 7f51caffd640 1 --2- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc09a630 0x7f51cc131160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.985+0000 7f51ca7fc640 1 --2- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09af50 0x7f51cc1316a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c390 tx=0x7f51c000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.985+0000 7f51b37fe640 1 -- 172.21.15.47:0/2182502753 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0017070 con 0x7f51cc09af50 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.985+0000 7f51b37fe640 1 -- 172.21.15.47:0/2182502753 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c00052d0 con 0x7f51cc09af50 2024-09-15T21:56:34.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.985+0000 7f51d1733640 1 -- 172.21.15.47:0/2182502753 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51cc132120 con 0x7f51cc09af50 2024-09-15T21:56:34.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.985+0000 7f51b37fe640 1 -- 172.21.15.47:0/2182502753 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0003830 con 0x7f51cc09af50 2024-09-15T21:56:34.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.985+0000 7f51d1733640 1 -- 172.21.15.47:0/2182502753 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f51cc133d90 con 0x7f51cc09af50 2024-09-15T21:56:34.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.986+0000 7f51b37fe640 1 -- 172.21.15.47:0/2182502753 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 8) ==== 55837+0+0 (secure 0 0 0) 0x7f51c001a070 con 0x7f51cc09af50 2024-09-15T21:56:34.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.986+0000 7f51d1733640 1 -- 172.21.15.47:0/2182502753 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51cc09aa10 con 0x7f51cc09af50 2024-09-15T21:56:34.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.987+0000 7f51b37fe640 1 -- 172.21.15.47:0/2182502753 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7f51c005ca30 con 0x7f51cc09af50 2024-09-15T21:56:34.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:34.991+0000 7f51b37fe640 1 -- 172.21.15.47:0/2182502753 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f51c003be70 con 0x7f51cc09af50 2024-09-15T21:56:35.004 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:56:35.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.078+0000 7fd528c67640 1 Processor -- start 2024-09-15T21:56:35.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.078+0000 7fd528c67640 1 -- start start 2024-09-15T21:56:35.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd528c67640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240983c0 0x7fd524097180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd528c67640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240976c0 0x7fd524097b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd528c67640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd524133da0 0x7fd524136190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd528c67640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd52406c060 con 0x7fd524133da0 2024-09-15T21:56:35.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd528c67640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd524096d80 con 0x7fd5240983c0 2024-09-15T21:56:35.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd528c67640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd5240991b0 con 0x7fd5240976c0 2024-09-15T21:56:35.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd521d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240976c0 0x7fd524097b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd521d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240976c0 0x7fd524097b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46610/0 (socket says 172.21.15.47:46610) 2024-09-15T21:56:35.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd52259c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240983c0 0x7fd524097180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd521d9b640 1 -- 172.21.15.47:0/1233774968 learned_addr learned my addr 172.21.15.47:0/1233774968 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:35.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.079+0000 7fd522d9d640 1 --2- 172.21.15.47:0/1233774968 >> v2:172.21.15.47:3300/0 conn(0x7fd524133da0 0x7fd524136190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.080+0000 7fd521d9b640 1 -- 172.21.15.47:0/1233774968 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240983c0 msgr2=0x7fd524097180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.080+0000 7fd521d9b640 1 --2- 172.21.15.47:0/1233774968 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240983c0 0x7fd524097180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.080+0000 7fd521d9b640 1 -- 172.21.15.47:0/1233774968 >> v2:172.21.15.47:3300/0 conn(0x7fd524133da0 msgr2=0x7fd524136190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.080+0000 7fd521d9b640 1 --2- 172.21.15.47:0/1233774968 >> v2:172.21.15.47:3300/0 conn(0x7fd524133da0 0x7fd524136190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.080+0000 7fd521d9b640 1 -- 172.21.15.47:0/1233774968 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5241366d0 con 0x7fd5240976c0 2024-09-15T21:56:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.081+0000 7fd521d9b640 1 --2- 172.21.15.47:0/1233774968 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240976c0 0x7fd524097b20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd518009870 tx=0x7fd51802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903f31ab88b17d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.081+0000 7fd52159a640 1 -- 172.21.15.47:0/1233774968 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518037070 con 0x7fd5240976c0 2024-09-15T21:56:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.081+0000 7fd52159a640 1 -- 172.21.15.47:0/1233774968 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd518032440 con 0x7fd5240976c0 2024-09-15T21:56:35.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.081+0000 7fd52159a640 1 -- 172.21.15.47:0/1233774968 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518032810 con 0x7fd5240976c0 2024-09-15T21:56:35.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.082+0000 7fd528c67640 1 -- 172.21.15.47:0/1233774968 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240976c0 msgr2=0x7fd524097b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.082+0000 7fd528c67640 1 --2- 172.21.15.47:0/1233774968 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240976c0 0x7fd524097b20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd518009870 tx=0x7fd51802a130 comp rx=0 tx=0).stop 2024-09-15T21:56:35.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.082+0000 7fd528c67640 1 -- 172.21.15.47:0/1233774968 shutdown_connections 2024-09-15T21:56:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.082+0000 7fd528c67640 1 --2- 172.21.15.47:0/1233774968 >> v2:172.21.15.47:3300/0 conn(0x7fd524133da0 0x7fd524136190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.082+0000 7fd528c67640 1 --2- 172.21.15.47:0/1233774968 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240976c0 0x7fd524097b20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.082+0000 7fd528c67640 1 --2- 172.21.15.47:0/1233774968 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240983c0 0x7fd524097180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.082+0000 7fd528c67640 1 -- 172.21.15.47:0/1233774968 >> 172.21.15.47:0/1233774968 conn(0x7fd52409fdf0 msgr2=0x7fd5240a0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.082+0000 7fd528c67640 1 -- 172.21.15.47:0/1233774968 shutdown_connections 2024-09-15T21:56:35.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.082+0000 7fd528c67640 1 -- 172.21.15.47:0/1233774968 wait complete. 2024-09-15T21:56:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.084+0000 7fd528c67640 1 Processor -- start 2024-09-15T21:56:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.084+0000 7fd528c67640 1 -- start start 2024-09-15T21:56:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.084+0000 7fd528c67640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240976c0 0x7fd524094d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.084+0000 7fd528c67640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240983c0 0x7fd5240952d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.084+0000 7fd528c67640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd524133da0 0x7fd524095810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.084+0000 7fd528c67640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd524136c30 con 0x7fd524133da0 2024-09-15T21:56:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.084+0000 7fd528c67640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd524136ab0 con 0x7fd5240976c0 2024-09-15T21:56:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.084+0000 7fd528c67640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd524136db0 con 0x7fd5240983c0 2024-09-15T21:56:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd52259c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240976c0 0x7fd524094d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd521d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240983c0 0x7fd5240952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd521d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240983c0 0x7fd5240952d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46626/0 (socket says 172.21.15.47:46626) 2024-09-15T21:56:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd521d9b640 1 -- 172.21.15.47:0/979685248 learned_addr learned my addr 172.21.15.47:0/979685248 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:35.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd522d9d640 1 --2- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd524133da0 0x7fd524095810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd521d9b640 1 -- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240976c0 msgr2=0x7fd524094d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd521d9b640 1 --2- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240976c0 0x7fd524094d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd521d9b640 1 -- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd524133da0 msgr2=0x7fd524095810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd521d9b640 1 --2- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd524133da0 0x7fd524095810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd521d9b640 1 -- 172.21.15.47:0/979685248 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd524073cb0 con 0x7fd5240983c0 2024-09-15T21:56:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.085+0000 7fd522d9d640 1 --2- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd524133da0 0x7fd524095810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.086+0000 7fd52259c640 1 --2- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240976c0 0x7fd524094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.086+0000 7fd521d9b640 1 --2- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240983c0 0x7fd5240952d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd51802a170 tx=0x7fd5180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.086+0000 7fd512ffd640 1 -- 172.21.15.47:0/979685248 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518041020 con 0x7fd5240983c0 2024-09-15T21:56:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.086+0000 7fd512ffd640 1 -- 172.21.15.47:0/979685248 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51803d620 con 0x7fd5240983c0 2024-09-15T21:56:35.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.086+0000 7fa4b9e38640 1 Processor -- start 2024-09-15T21:56:35.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.086+0000 7fd528c67640 1 -- 172.21.15.47:0/979685248 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52406e900 con 0x7fd5240983c0 2024-09-15T21:56:35.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.086+0000 7fd512ffd640 1 -- 172.21.15.47:0/979685248 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518037040 con 0x7fd5240983c0 2024-09-15T21:56:35.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.086+0000 7fd528c67640 1 -- 172.21.15.47:0/979685248 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52406f470 con 0x7fd5240983c0 2024-09-15T21:56:35.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.086+0000 7fa4b9e38640 1 -- start start 2024-09-15T21:56:35.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b9e38640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa4b412c470 0x7fa4b412c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fd512ffd640 1 -- 172.21.15.47:0/979685248 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 8) ==== 55837+0+0 (secure 0 0 0) 0x7fd518044020 con 0x7fd5240983c0 2024-09-15T21:56:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b9e38640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b412cd90 0x7fa4b4128f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b9e38640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b41294b0 0x7fa4b4129930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b9e38640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b4075800 con 0x7fa4b412c470 2024-09-15T21:56:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b9e38640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b406e5a0 con 0x7fa4b412cd90 2024-09-15T21:56:35.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b9e38640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fa4b4098930 con 0x7fa4b41294b0 2024-09-15T21:56:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b2ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b412cd90 0x7fa4b4128f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b3fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b41294b0 0x7fa4b4129930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b2ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b412cd90 0x7fa4b4128f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40064/0 (socket says 172.21.15.47:40064) 2024-09-15T21:56:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b2ffd640 1 -- 172.21.15.47:0/3721710918 learned_addr learned my addr 172.21.15.47:0/3721710918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.087+0000 7fa4b37fe640 1 --2- 172.21.15.47:0/3721710918 >> v2:172.21.15.47:3300/0 conn(0x7fa4b412c470 0x7fa4b412c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.088+0000 7fa4b2ffd640 1 -- 172.21.15.47:0/3721710918 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b41294b0 msgr2=0x7fa4b4129930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.088+0000 7fa4b2ffd640 1 --2- 172.21.15.47:0/3721710918 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b41294b0 0x7fa4b4129930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.088+0000 7fd512ffd640 1 -- 172.21.15.47:0/979685248 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7fd51807dca0 con 0x7fd5240983c0 2024-09-15T21:56:35.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.088+0000 7fa4b2ffd640 1 -- 172.21.15.47:0/3721710918 >> v2:172.21.15.47:3300/0 conn(0x7fa4b412c470 msgr2=0x7fa4b412c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:35.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.088+0000 7fa4b2ffd640 1 --2- 172.21.15.47:0/3721710918 >> v2:172.21.15.47:3300/0 conn(0x7fa4b412c470 0x7fa4b412c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.088+0000 7fa4b2ffd640 1 -- 172.21.15.47:0/3721710918 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b412d1b0 con 0x7fa4b412cd90 2024-09-15T21:56:35.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.088+0000 7fd528c67640 1 -- 172.21.15.47:0/979685248 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd524097b20 con 0x7fd5240983c0 2024-09-15T21:56:35.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.089+0000 7fa4b2ffd640 1 --2- 172.21.15.47:0/3721710918 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b412cd90 0x7fa4b4128f70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009870 tx=0x7fa4a802a090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6399450c38a58ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:35.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.089+0000 7fa4b27fc640 1 -- 172.21.15.47:0/3721710918 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8037070 con 0x7fa4b412cd90 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.089+0000 7fa4b27fc640 1 -- 172.21.15.47:0/3721710918 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a802ade0 con 0x7fa4b412cd90 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.089+0000 7fa4b27fc640 1 -- 172.21.15.47:0/3721710918 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8032660 con 0x7fa4b412cd90 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.090+0000 7fa4b9e38640 1 -- 172.21.15.47:0/3721710918 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b412cd90 msgr2=0x7fa4b4128f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.090+0000 7fa4b9e38640 1 --2- 172.21.15.47:0/3721710918 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b412cd90 0x7fa4b4128f70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009870 tx=0x7fa4a802a090 comp rx=0 tx=0).stop 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.090+0000 7fa4b9e38640 1 -- 172.21.15.47:0/3721710918 shutdown_connections 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.090+0000 7fa4b9e38640 1 --2- 172.21.15.47:0/3721710918 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b41294b0 0x7fa4b4129930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.090+0000 7fa4b9e38640 1 --2- 172.21.15.47:0/3721710918 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b412cd90 0x7fa4b4128f70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.090+0000 7fa4b9e38640 1 --2- 172.21.15.47:0/3721710918 >> v2:172.21.15.47:3300/0 conn(0x7fa4b412c470 0x7fa4b412c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.090+0000 7fa4b9e38640 1 -- 172.21.15.47:0/3721710918 >> 172.21.15.47:0/3721710918 conn(0x7fa4b409b880 msgr2=0x7fa4b409bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.090+0000 7fa4b9e38640 1 -- 172.21.15.47:0/3721710918 shutdown_connections 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.090+0000 7fa4b9e38640 1 -- 172.21.15.47:0/3721710918 wait complete. 2024-09-15T21:56:35.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.091+0000 7fa4b9e38640 1 Processor -- start 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.091+0000 7fa4b9e38640 1 -- start start 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.092+0000 7fa4b9e38640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b41294b0 0x7fa4b41c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.092+0000 7fa4b9e38640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4b412c470 0x7fa4b41c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.092+0000 7fa4b9e38640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b412cd90 0x7fa4b4125db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.092+0000 7fa4b9e38640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b406cf40 con 0x7fa4b412c470 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.092+0000 7fa4b9e38640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b412d3e0 con 0x7fa4b41294b0 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.092+0000 7fa4b9e38640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fa4b406d0c0 con 0x7fa4b412cd90 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.092+0000 7fa4b37fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b41294b0 0x7fa4b41c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.092+0000 7fa4b37fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b41294b0 0x7fa4b41c05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40072/0 (socket says 172.21.15.47:40072) 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.092+0000 7fa4b37fe640 1 -- 172.21.15.47:0/2359009299 learned_addr learned my addr 172.21.15.47:0/2359009299 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b3fff640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b412cd90 0x7fa4b4125db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b2ffd640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4b412c470 0x7fa4b41c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b37fe640 1 -- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b412cd90 msgr2=0x7fa4b4125db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b37fe640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b412cd90 0x7fa4b4125db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b37fe640 1 -- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4b412c470 msgr2=0x7fa4b41c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b37fe640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4b412c470 0x7fa4b41c0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b37fe640 1 -- 172.21.15.47:0/2359009299 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b4074130 con 0x7fa4b41294b0 2024-09-15T21:56:35.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b3fff640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b412cd90 0x7fa4b4125db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:35.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b2ffd640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4b412c470 0x7fa4b41c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:56:35.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b37fe640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b41294b0 0x7fa4b41c05c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009410 tx=0x7fa4a40098d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:35.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa49bfff640 1 -- 172.21.15.47:0/2359009299 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4022070 con 0x7fa4b41294b0 2024-09-15T21:56:35.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fd512ffd640 1 -- 172.21.15.47:0/979685248 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fd51804b030 con 0x7fd5240983c0 2024-09-15T21:56:35.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa49bfff640 1 -- 172.21.15.47:0/2359009299 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a40070d0 con 0x7fa4b41294b0 2024-09-15T21:56:35.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa49bfff640 1 -- 172.21.15.47:0/2359009299 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40074a0 con 0x7fa4b41294b0 2024-09-15T21:56:35.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b9e38640 1 -- 172.21.15.47:0/2359009299 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b41289b0 con 0x7fa4b41294b0 2024-09-15T21:56:35.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.093+0000 7fa4b9e38640 1 -- 172.21.15.47:0/2359009299 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b41c1040 con 0x7fa4b41294b0 2024-09-15T21:56:35.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.094+0000 7fa49bfff640 1 -- 172.21.15.47:0/2359009299 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 8) ==== 55837+0+0 (secure 0 0 0) 0x7fa4a4025070 con 0x7fa4b41294b0 2024-09-15T21:56:35.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.094+0000 7fa4b9e38640 1 -- 172.21.15.47:0/2359009299 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b412c850 con 0x7fa4b41294b0 2024-09-15T21:56:35.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.102+0000 7fa49bfff640 1 -- 172.21.15.47:0/2359009299 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7fa4a4065c00 con 0x7fa4b41294b0 2024-09-15T21:56:35.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.102+0000 7fa49bfff640 1 -- 172.21.15.47:0/2359009299 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fa4a406b1f0 con 0x7fa4b41294b0 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.116+0000 7f1d56248640 1 Processor -- start 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.117+0000 7f1d56248640 1 -- start start 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.117+0000 7f1d56248640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d5009b120 0x7f1d50134720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.117+0000 7f1d56248640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d50134c60 0x7f1d50137050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.117+0000 7f1d56248640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d5009b600 con 0x7f1d5009a630 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.117+0000 7f1d56248640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1d5006d480 con 0x7f1d5009b120 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.117+0000 7f1d56248640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f1d5006d140 con 0x7f1d50134c60 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.117+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d50134c60 0x7f1d50137050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.117+0000 7f1d4f7fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1d5009a630 0x7f1d5009aa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36934/0 (socket says 172.21.15.47:36934) 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.117+0000 7f1d4f7fe640 1 -- 172.21.15.47:0/2070680279 learned_addr learned my addr 172.21.15.47:0/2070680279 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:35.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.117+0000 7f1d4effd640 1 --2- 172.21.15.47:0/2070680279 >> v2:172.21.15.47:3300/0 conn(0x7f1d5009b120 0x7f1d50134720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.118+0000 7f1d4effd640 1 -- 172.21.15.47:0/2070680279 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d50134c60 msgr2=0x7f1d50137050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.118+0000 7f1d4effd640 1 --2- 172.21.15.47:0/2070680279 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d50134c60 0x7f1d50137050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.118+0000 7f1d4effd640 1 -- 172.21.15.47:0/2070680279 >> v1:172.21.15.47:6789/0 conn(0x7f1d5009a630 legacy=0x7f1d5009aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.118+0000 7f1d4effd640 1 -- 172.21.15.47:0/2070680279 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5009bc00 con 0x7f1d5009b120 2024-09-15T21:56:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.118+0000 7f1d4ffff640 1 --2- 172.21.15.47:0/2070680279 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d50134c60 0x7f1d50137050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.119+0000 7f1d4effd640 1 --2- 172.21.15.47:0/2070680279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d5009b120 0x7f1d50134720 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d440097a0 tx=0x7f1d4402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a5d157f20f3454 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.119+0000 7f1d4e7fc640 1 -- 172.21.15.47:0/2070680279 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44037070 con 0x7f1d5009b120 2024-09-15T21:56:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.119+0000 7f1d4e7fc640 1 -- 172.21.15.47:0/2070680279 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4402adb0 con 0x7f1d5009b120 2024-09-15T21:56:35.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.119+0000 7f1d4e7fc640 1 -- 172.21.15.47:0/2070680279 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44032660 con 0x7f1d5009b120 2024-09-15T21:56:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.120+0000 7f1d56248640 1 -- 172.21.15.47:0/2070680279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d5009b120 msgr2=0x7f1d50134720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.120+0000 7f1d56248640 1 --2- 172.21.15.47:0/2070680279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d5009b120 0x7f1d50134720 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d440097a0 tx=0x7f1d4402a060 comp rx=0 tx=0).stop 2024-09-15T21:56:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.120+0000 7f1d56248640 1 -- 172.21.15.47:0/2070680279 shutdown_connections 2024-09-15T21:56:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.120+0000 7f1d56248640 1 --2- 172.21.15.47:0/2070680279 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d50134c60 0x7f1d50137050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.120+0000 7f1d56248640 1 --2- 172.21.15.47:0/2070680279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d5009b120 0x7f1d50134720 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.120+0000 7f1d56248640 1 -- 172.21.15.47:0/2070680279 >> 172.21.15.47:0/2070680279 conn(0x7f1d5009eb90 msgr2=0x7f1d5009efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.120+0000 7f1d56248640 1 -- 172.21.15.47:0/2070680279 shutdown_connections 2024-09-15T21:56:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.120+0000 7f1d56248640 1 -- 172.21.15.47:0/2070680279 wait complete. 2024-09-15T21:56:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.121+0000 7f1d56248640 1 Processor -- start 2024-09-15T21:56:35.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.121+0000 7f1d56248640 1 -- start start 2024-09-15T21:56:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.121+0000 7f1d56248640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d5009a630 0x7f1d50126f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.121+0000 7f1d56248640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d5009b120 0x7f1d501274b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.121+0000 7f1d56248640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d50134c60 0x7f1d501279f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4effd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d5009b120 0x7f1d501274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d50134c60 0x7f1d501279f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d50134c60 0x7f1d501279f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47848/0 (socket says 172.21.15.47:47848) 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4effd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d5009b120 0x7f1d501274b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46678/0 (socket says 172.21.15.47:46678) 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4ffff640 1 -- 172.21.15.47:0/1858616297 learned_addr learned my addr 172.21.15.47:0/1858616297 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50137920 con 0x7f1d50134c60 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4f7fe640 1 --2- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d5009a630 0x7f1d50126f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f1d501377a0 con 0x7f1d5009a630 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f1d50137aa0 con 0x7f1d5009b120 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4ffff640 1 -- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d5009b120 msgr2=0x7f1d501274b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4ffff640 1 --2- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d5009b120 0x7f1d501274b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4ffff640 1 -- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d5009a630 msgr2=0x7f1d50126f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4ffff640 1 --2- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d5009a630 0x7f1d50126f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4ffff640 1 -- 172.21.15.47:0/1858616297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d50129240 con 0x7f1d50134c60 2024-09-15T21:56:35.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.122+0000 7f1d4f7fe640 1 --2- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d5009a630 0x7f1d50126f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:35.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.123+0000 7f1d4effd640 1 --2- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d5009b120 0x7f1d501274b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:35.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.123+0000 7f1d4ffff640 1 --2- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d50134c60 0x7f1d501279f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800efb0 tx=0x7f1d4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:35.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.123+0000 7f1d37fff640 1 -- 172.21.15.47:0/1858616297 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48018020 con 0x7f1d50134c60 2024-09-15T21:56:35.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.123+0000 7f1d37fff640 1 -- 172.21.15.47:0/1858616297 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d48014480 con 0x7f1d50134c60 2024-09-15T21:56:35.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.123+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d50128dd0 con 0x7f1d50134c60 2024-09-15T21:56:35.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.123+0000 7f1d37fff640 1 -- 172.21.15.47:0/1858616297 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48014850 con 0x7f1d50134c60 2024-09-15T21:56:35.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.123+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d50134380 con 0x7f1d50134c60 2024-09-15T21:56:35.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.125+0000 7f1d37fff640 1 -- 172.21.15.47:0/1858616297 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 55837+0+0 (secure 0 0 0) 0x7f1d48005d10 con 0x7f1d50134c60 2024-09-15T21:56:35.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.125+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d5009aa10 con 0x7f1d50134c60 2024-09-15T21:56:35.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.125+0000 7f1d37fff640 1 -- 172.21.15.47:0/1858616297 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7f1d4805e420 con 0x7f1d50134c60 2024-09-15T21:56:35.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.133+0000 7f1d37fff640 1 -- 172.21.15.47:0/1858616297 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f1d48022030 con 0x7f1d50134c60 2024-09-15T21:56:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.185+0000 7f51d1733640 1 -- 172.21.15.47:0/2182502753 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f51cc075c30 con 0x7f51cc09af50 2024-09-15T21:56:35.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.186+0000 7f51b37fe640 1 -- 172.21.15.47:0/2182502753 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f51c005ce20 con 0x7f51cc09af50 2024-09-15T21:56:35.187 INFO:teuthology.orchestra.run.smithi047.stdout:38654705682 2024-09-15T21:56:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.188+0000 7f51d1733640 1 -- 172.21.15.47:0/2182502753 >> v1:172.21.15.47:6800/1790418198 conn(0x7f51b8047e20 legacy=0x7f51b804a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.188+0000 7f51d1733640 1 -- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09af50 msgr2=0x7f51cc1316a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.188+0000 7f51d1733640 1 --2- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09af50 0x7f51cc1316a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c390 tx=0x7f51c000c850 comp rx=0 tx=0).stop 2024-09-15T21:56:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.188+0000 7f51d1733640 1 -- 172.21.15.47:0/2182502753 shutdown_connections 2024-09-15T21:56:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.188+0000 7f51d1733640 1 --2- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51cc12faf0 0x7f51cc131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.188+0000 7f51d1733640 1 --2- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f51cc09af50 0x7f51cc1316a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.188+0000 7f51d1733640 1 --2- 172.21.15.47:0/2182502753 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f51cc09a630 0x7f51cc131160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.188+0000 7f51d1733640 1 -- 172.21.15.47:0/2182502753 >> 172.21.15.47:0/2182502753 conn(0x7f51cc09eb90 msgr2=0x7f51cc12e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.188+0000 7f51d1733640 1 -- 172.21.15.47:0/2182502753 shutdown_connections 2024-09-15T21:56:35.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.188+0000 7f51d1733640 1 -- 172.21.15.47:0/2182502753 wait complete. 2024-09-15T21:56:35.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705684 got 38654705682 for osd.3 2024-09-15T21:56:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.259+0000 7fa4b9e38640 1 -- 172.21.15.47:0/2359009299 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa4b4068d50 con 0x7fa4b41294b0 2024-09-15T21:56:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.260+0000 7fa49bfff640 1 -- 172.21.15.47:0/2359009299 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa4a4044f70 con 0x7fa4b41294b0 2024-09-15T21:56:35.261 INFO:teuthology.orchestra.run.smithi047.stdout:38654705682 2024-09-15T21:56:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.261+0000 7fa4b9e38640 1 -- 172.21.15.47:0/2359009299 >> v1:172.21.15.47:6800/1790418198 conn(0x7fa4a0047e50 legacy=0x7fa4a004a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.261+0000 7fa4b9e38640 1 -- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b41294b0 msgr2=0x7fa4b41c05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.261+0000 7fa4b9e38640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b41294b0 0x7fa4b41c05c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009410 tx=0x7fa4a40098d0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.261+0000 7fa4b9e38640 1 -- 172.21.15.47:0/2359009299 shutdown_connections 2024-09-15T21:56:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.261+0000 7fa4b9e38640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa4b412cd90 0x7fa4b4125db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.261+0000 7fa4b9e38640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4b412c470 0x7fa4b41c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.261+0000 7fa4b9e38640 1 --2- 172.21.15.47:0/2359009299 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa4b41294b0 0x7fa4b41c05c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.261+0000 7fa4b9e38640 1 -- 172.21.15.47:0/2359009299 >> 172.21.15.47:0/2359009299 conn(0x7fa4b409b880 msgr2=0x7fa4b406eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.261+0000 7fa4b9e38640 1 -- 172.21.15.47:0/2359009299 shutdown_connections 2024-09-15T21:56:35.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.261+0000 7fa4b9e38640 1 -- 172.21.15.47:0/2359009299 wait complete. 2024-09-15T21:56:35.273 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705684 got 38654705682 for osd.2 2024-09-15T21:56:35.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.282+0000 7fd528c67640 1 -- 172.21.15.47:0/979685248 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd524075b40 con 0x7fd5240983c0 2024-09-15T21:56:35.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.283+0000 7fd512ffd640 1 -- 172.21.15.47:0/979685248 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd51807cae0 con 0x7fd5240983c0 2024-09-15T21:56:35.284 INFO:teuthology.orchestra.run.smithi047.stdout:38654705681 2024-09-15T21:56:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.284+0000 7fd528c67640 1 -- 172.21.15.47:0/979685248 >> v1:172.21.15.47:6800/1790418198 conn(0x7fd504047e20 legacy=0x7fd50404a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.284+0000 7fd528c67640 1 -- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240983c0 msgr2=0x7fd5240952d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.284+0000 7fd528c67640 1 --2- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240983c0 0x7fd5240952d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd51802a170 tx=0x7fd5180046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.284+0000 7fd528c67640 1 -- 172.21.15.47:0/979685248 shutdown_connections 2024-09-15T21:56:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.284+0000 7fd528c67640 1 --2- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd524133da0 0x7fd524095810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.284+0000 7fd528c67640 1 --2- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd5240983c0 0x7fd5240952d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.284+0000 7fd528c67640 1 --2- 172.21.15.47:0/979685248 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd5240976c0 0x7fd524094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.284+0000 7fd528c67640 1 -- 172.21.15.47:0/979685248 >> 172.21.15.47:0/979685248 conn(0x7fd52409fdf0 msgr2=0x7fd524135480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.285+0000 7fd528c67640 1 -- 172.21.15.47:0/979685248 shutdown_connections 2024-09-15T21:56:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.285+0000 7fd528c67640 1 -- 172.21.15.47:0/979685248 wait complete. 2024-09-15T21:56:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.290+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1d50130010 con 0x7f1d50134c60 2024-09-15T21:56:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.290+0000 7f1d37fff640 1 -- 172.21.15.47:0/1858616297 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1d50129240 con 0x7f1d50134c60 2024-09-15T21:56:35.292 INFO:teuthology.orchestra.run.smithi047.stdout:38654705681 2024-09-15T21:56:35.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.292+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 >> v1:172.21.15.47:6800/1790418198 conn(0x7f1d3c047e50 legacy=0x7f1d3c04a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.292+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d50134c60 msgr2=0x7f1d501279f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:35.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.292+0000 7f1d56248640 1 --2- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d50134c60 0x7f1d501279f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800efb0 tx=0x7f1d4800c490 comp rx=0 tx=0).stop 2024-09-15T21:56:35.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.292+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 shutdown_connections 2024-09-15T21:56:35.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.292+0000 7f1d56248640 1 --2- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d50134c60 0x7f1d501279f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.292+0000 7f1d56248640 1 --2- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d5009b120 0x7f1d501274b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.292+0000 7f1d56248640 1 --2- 172.21.15.47:0/1858616297 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d5009a630 0x7f1d50126f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:35.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.292+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 >> 172.21.15.47:0/1858616297 conn(0x7f1d5009eb90 msgr2=0x7f1d5012d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:35.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.292+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 shutdown_connections 2024-09-15T21:56:35.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:35.292+0000 7f1d56248640 1 -- 172.21.15.47:0/1858616297 wait complete. 2024-09-15T21:56:35.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.1 2024-09-15T21:56:35.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705681 for osd.0 2024-09-15T21:56:36.206 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:56:36.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:56:36.298 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:56:36.306 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:56:36.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.320+0000 7fee0500d640 1 Processor -- start 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.320+0000 7fee0500d640 1 -- start start 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.320+0000 7fee0500d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee0012c130 0x7fee0012c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.320+0000 7fee0500d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee0009c000 0x7fee0012ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.320+0000 7fee0500d640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee000982e0 con 0x7fee00095060 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.320+0000 7fee0500d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fee0006e2b0 con 0x7fee0009c000 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.320+0000 7fee0500d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fee0006df20 con 0x7fee0012c130 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.321+0000 7fedfed9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee0012c130 0x7fee0012c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.321+0000 7fedfed9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee0012c130 0x7fee0012c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46702/0 (socket says 172.21.15.47:46702) 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.321+0000 7fedfed9d640 1 -- 172.21.15.47:0/3583977399 learned_addr learned my addr 172.21.15.47:0/3583977399 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.321+0000 7fedfe59c640 1 --2- 172.21.15.47:0/3583977399 >> v2:172.21.15.47:3300/0 conn(0x7fee0009c000 0x7fee0012ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.321+0000 7fedfed9d640 1 -- 172.21.15.47:0/3583977399 >> v2:172.21.15.47:3300/0 conn(0x7fee0009c000 msgr2=0x7fee0012ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.321+0000 7fedfed9d640 1 --2- 172.21.15.47:0/3583977399 >> v2:172.21.15.47:3300/0 conn(0x7fee0009c000 0x7fee0012ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.321+0000 7fedfed9d640 1 -- 172.21.15.47:0/3583977399 >> v1:172.21.15.47:6789/0 conn(0x7fee00095060 legacy=0x7fee000954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.321+0000 7fedfed9d640 1 -- 172.21.15.47:0/3583977399 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee00098c20 con 0x7fee0012c130 2024-09-15T21:56:36.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.321+0000 7fedfe59c640 1 --2- 172.21.15.47:0/3583977399 >> v2:172.21.15.47:3300/0 conn(0x7fee0009c000 0x7fee0012ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.322+0000 7fedfed9d640 1 --2- 172.21.15.47:0/3583977399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee0012c130 0x7fee0012c510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009870 tx=0x7fedf002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2138a7ad982ee607 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.322+0000 7fedfdd9b640 1 -- 172.21.15.47:0/3583977399 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0037070 con 0x7fee0012c130 2024-09-15T21:56:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.323+0000 7fedfdd9b640 1 -- 172.21.15.47:0/3583977399 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf0032440 con 0x7fee0012c130 2024-09-15T21:56:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.323+0000 7fedfdd9b640 1 -- 172.21.15.47:0/3583977399 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0032810 con 0x7fee0012c130 2024-09-15T21:56:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.323+0000 7fee0500d640 1 -- 172.21.15.47:0/3583977399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee0012c130 msgr2=0x7fee0012c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.323+0000 7fee0500d640 1 --2- 172.21.15.47:0/3583977399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee0012c130 0x7fee0012c510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009870 tx=0x7fedf002a130 comp rx=0 tx=0).stop 2024-09-15T21:56:36.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.323+0000 7fee0500d640 1 -- 172.21.15.47:0/3583977399 shutdown_connections 2024-09-15T21:56:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.323+0000 7fee0500d640 1 --2- 172.21.15.47:0/3583977399 >> v2:172.21.15.47:3300/0 conn(0x7fee0009c000 0x7fee0012ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.323+0000 7fee0500d640 1 --2- 172.21.15.47:0/3583977399 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee0012c130 0x7fee0012c510 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.323+0000 7fee0500d640 1 -- 172.21.15.47:0/3583977399 >> 172.21.15.47:0/3583977399 conn(0x7fee0009f2a0 msgr2=0x7fee0009b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.324+0000 7fee0500d640 1 -- 172.21.15.47:0/3583977399 shutdown_connections 2024-09-15T21:56:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.324+0000 7fee0500d640 1 -- 172.21.15.47:0/3583977399 wait complete. 2024-09-15T21:56:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.325+0000 7fee0500d640 1 Processor -- start 2024-09-15T21:56:36.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.325+0000 7fee0500d640 1 -- start start 2024-09-15T21:56:36.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.325+0000 7fee0500d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee00095060 0x7fee00135690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.325+0000 7fee0500d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee0009c000 0x7fee00135bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fee0500d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee0012c130 0x7fee00136110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedfed9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee00095060 0x7fee00135690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedfed9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee00095060 0x7fee00135690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46714/0 (socket says 172.21.15.47:46714) 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedfed9d640 1 -- 172.21.15.47:0/2146310303 learned_addr learned my addr 172.21.15.47:0/2146310303 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fee00134680 con 0x7fee0012c130 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fee00095e50 con 0x7fee0009c000 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fee00134800 con 0x7fee00095060 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedfe59c640 1 --2- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee0009c000 0x7fee00135bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedff59e640 1 --2- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee0012c130 0x7fee00136110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedfe59c640 1 -- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee00095060 msgr2=0x7fee00135690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedfe59c640 1 --2- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee00095060 0x7fee00135690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedfe59c640 1 -- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee0012c130 msgr2=0x7fee00136110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedfe59c640 1 --2- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee0012c130 0x7fee00136110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedfe59c640 1 -- 172.21.15.47:0/2146310303 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee00128da0 con 0x7fee0009c000 2024-09-15T21:56:36.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.326+0000 7fedfed9d640 1 --2- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee00095060 0x7fee00135690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.327+0000 7fedfe59c640 1 --2- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee0009c000 0x7fee00135bd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fedf400b170 tx=0x7fedf400b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.327+0000 7fede77fe640 1 -- 172.21.15.47:0/2146310303 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4022070 con 0x7fee0009c000 2024-09-15T21:56:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.327+0000 7fede77fe640 1 -- 172.21.15.47:0/2146310303 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf401b520 con 0x7fee0009c000 2024-09-15T21:56:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.327+0000 7fede77fe640 1 -- 172.21.15.47:0/2146310303 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf400be60 con 0x7fee0009c000 2024-09-15T21:56:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.327+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee00073aa0 con 0x7fee0009c000 2024-09-15T21:56:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.327+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fee001c4630 con 0x7fee0009c000 2024-09-15T21:56:36.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.328+0000 7fede77fe640 1 -- 172.21.15.47:0/2146310303 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 9) ==== 55943+0+0 (secure 0 0 0) 0x7fedf4025070 con 0x7fee0009c000 2024-09-15T21:56:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.328+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee000954f0 con 0x7fee0009c000 2024-09-15T21:56:36.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.328+0000 7fede77fe640 1 -- 172.21.15.47:0/2146310303 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7fedf4065600 con 0x7fee0009c000 2024-09-15T21:56:36.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.334+0000 7fede77fe640 1 -- 172.21.15.47:0/2146310303 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fedf4044870 con 0x7fee0009c000 2024-09-15T21:56:36.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.386+0000 7fb524153640 1 Processor -- start 2024-09-15T21:56:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.386+0000 7fb524153640 1 -- start start 2024-09-15T21:56:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb524153640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0983c0 0x7fb51c097180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb524153640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb51c0976c0 0x7fb51c097b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb524153640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb51c096d80 con 0x7fb51c133da0 2024-09-15T21:56:36.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb524153640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb51c06c060 con 0x7fb51c0976c0 2024-09-15T21:56:36.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb524153640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fb51c0991b0 con 0x7fb51c0983c0 2024-09-15T21:56:36.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb521eef640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0983c0 0x7fb51c097180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb521eef640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0983c0 0x7fb51c097180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40130/0 (socket says 172.21.15.47:40130) 2024-09-15T21:56:36.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb521eef640 1 -- 172.21.15.47:0/1051581187 learned_addr learned my addr 172.21.15.47:0/1051581187 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:36.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb5216ee640 1 --2- 172.21.15.47:0/1051581187 >> v2:172.21.15.47:3300/0 conn(0x7fb51c0976c0 0x7fb51c097b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb5216ee640 1 -- 172.21.15.47:0/1051581187 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0983c0 msgr2=0x7fb51c097180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb5216ee640 1 --2- 172.21.15.47:0/1051581187 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0983c0 0x7fb51c097180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb5216ee640 1 -- 172.21.15.47:0/1051581187 >> v1:172.21.15.47:6789/0 conn(0x7fb51c133da0 legacy=0x7fb51c136190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.387+0000 7fb5216ee640 1 -- 172.21.15.47:0/1051581187 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c1388c0 con 0x7fb51c0976c0 2024-09-15T21:56:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.388+0000 7fb5216ee640 1 --2- 172.21.15.47:0/1051581187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51c0976c0 0x7fb51c097b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0097a0 tx=0x7fb50c02a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75104b372d5ff9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.388+0000 7fb520eed640 1 -- 172.21.15.47:0/1051581187 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c037070 con 0x7fb51c0976c0 2024-09-15T21:56:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.388+0000 7fb520eed640 1 -- 172.21.15.47:0/1051581187 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c02ae40 con 0x7fb51c0976c0 2024-09-15T21:56:36.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.388+0000 7fb520eed640 1 -- 172.21.15.47:0/1051581187 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c032660 con 0x7fb51c0976c0 2024-09-15T21:56:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.389+0000 7fb524153640 1 -- 172.21.15.47:0/1051581187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51c0976c0 msgr2=0x7fb51c097b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.389+0000 7fb524153640 1 --2- 172.21.15.47:0/1051581187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51c0976c0 0x7fb51c097b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0097a0 tx=0x7fb50c02a0f0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.389+0000 7fb524153640 1 -- 172.21.15.47:0/1051581187 shutdown_connections 2024-09-15T21:56:36.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.389+0000 7fb524153640 1 --2- 172.21.15.47:0/1051581187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51c0976c0 0x7fb51c097b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.389+0000 7fb524153640 1 --2- 172.21.15.47:0/1051581187 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0983c0 0x7fb51c097180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.389+0000 7fb524153640 1 -- 172.21.15.47:0/1051581187 >> 172.21.15.47:0/1051581187 conn(0x7fb51c09fdf0 msgr2=0x7fb51c0a0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.390+0000 7fb524153640 1 -- 172.21.15.47:0/1051581187 shutdown_connections 2024-09-15T21:56:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.390+0000 7fb524153640 1 -- 172.21.15.47:0/1051581187 wait complete. 2024-09-15T21:56:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.391+0000 7fb524153640 1 Processor -- start 2024-09-15T21:56:36.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.391+0000 7fb524153640 1 -- start start 2024-09-15T21:56:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.391+0000 7fb524153640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0976c0 0x7fb51c070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.391+0000 7fb524153640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb51c0983c0 0x7fb51c070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.391+0000 7fb524153640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51c133da0 0x7fb51c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.391+0000 7fb524153640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb51c137c80 con 0x7fb51c133da0 2024-09-15T21:56:36.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.391+0000 7fb524153640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fb51c137b00 con 0x7fb51c0976c0 2024-09-15T21:56:36.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.391+0000 7fb524153640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fb51c137e00 con 0x7fb51c0983c0 2024-09-15T21:56:36.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5216ee640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb51c0983c0 0x7fb51c070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5226f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51c133da0 0x7fb51c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb521eef640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0976c0 0x7fb51c070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5216ee640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb51c0983c0 0x7fb51c070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46726/0 (socket says 172.21.15.47:46726) 2024-09-15T21:56:36.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5216ee640 1 -- 172.21.15.47:0/704660753 learned_addr learned my addr 172.21.15.47:0/704660753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5226f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51c133da0 0x7fb51c0710c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47918/0 (socket says 172.21.15.47:47918) 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5216ee640 1 -- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0976c0 msgr2=0x7fb51c070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5216ee640 1 --2- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0976c0 0x7fb51c070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5216ee640 1 -- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51c133da0 msgr2=0x7fb51c0710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5216ee640 1 --2- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51c133da0 0x7fb51c0710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5216ee640 1 -- 172.21.15.47:0/704660753 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51c072950 con 0x7fb51c0983c0 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb521eef640 1 --2- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0976c0 0x7fb51c070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.392+0000 7fb5216ee640 1 --2- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb51c0983c0 0x7fb51c070b80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb50c035040 tx=0x7fb50c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.393+0000 7fb5127fc640 1 -- 172.21.15.47:0/704660753 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c041070 con 0x7fb51c0983c0 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.393+0000 7fb5127fc640 1 -- 172.21.15.47:0/704660753 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c03c460 con 0x7fb51c0983c0 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.393+0000 7fb524153640 1 -- 172.21.15.47:0/704660753 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51c133850 con 0x7fb51c0983c0 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.393+0000 7fb5127fc640 1 -- 172.21.15.47:0/704660753 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c037040 con 0x7fb51c0983c0 2024-09-15T21:56:36.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.393+0000 7fb524153640 1 -- 172.21.15.47:0/704660753 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51c073250 con 0x7fb51c0983c0 2024-09-15T21:56:36.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.394+0000 7fb524153640 1 -- 172.21.15.47:0/704660753 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb51c138330 con 0x7fb51c0983c0 2024-09-15T21:56:36.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.395+0000 7fb5127fc640 1 -- 172.21.15.47:0/704660753 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 9) ==== 55943+0+0 (secure 0 0 0) 0x7fb50c03a070 con 0x7fb51c0983c0 2024-09-15T21:56:36.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.396+0000 7fb5127fc640 1 -- 172.21.15.47:0/704660753 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7fb50c07df20 con 0x7fb51c0983c0 2024-09-15T21:56:36.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.399+0000 7fb5127fc640 1 -- 172.21.15.47:0/704660753 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fb51c138330 con 0x7fb51c0983c0 2024-09-15T21:56:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.427+0000 7fd2f43e4640 1 Processor -- start 2024-09-15T21:56:36.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.427+0000 7fd2f43e4640 1 -- start start 2024-09-15T21:56:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.427+0000 7feb7d261640 1 Processor -- start 2024-09-15T21:56:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.427+0000 7feb7d261640 1 -- start start 2024-09-15T21:56:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.427+0000 7fd2f43e4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2ec097890 0x7fd2ec134db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.427+0000 7fd2f43e4640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097d20 0x7fd2ec137300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f43e4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2ec096d80 con 0x7fd2ec0983c0 2024-09-15T21:56:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f43e4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd2ec06c060 con 0x7fd2ec097890 2024-09-15T21:56:36.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f43e4640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd2ec0991b0 con 0x7fd2ec097d20 2024-09-15T21:56:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.427+0000 7feb7d261640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097b70 0x7feb78096110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.427+0000 7feb7d261640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb78096650 0x7feb78137900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb76ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feb78097790 0x7feb78095a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36956/0 (socket says 172.21.15.47:36956) 2024-09-15T21:56:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f2981640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097d20 0x7fd2ec137300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f197f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd2ec097890 0x7fd2ec134db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f2981640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097d20 0x7fd2ec137300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46736/0 (socket says 172.21.15.47:46736) 2024-09-15T21:56:36.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f2981640 1 -- 172.21.15.47:0/2896567408 learned_addr learned my addr 172.21.15.47:0/2896567408 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb767fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097b70 0x7feb78096110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb767fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097b70 0x7feb78096110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40142/0 (socket says 172.21.15.47:40142) 2024-09-15T21:56:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb76ffd640 1 -- 172.21.15.47:0/3960630526 learned_addr learned my addr 172.21.15.47:0/3960630526 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb777fe640 1 --2- 172.21.15.47:0/3960630526 >> v2:172.21.15.47:3300/0 conn(0x7feb78096650 0x7feb78137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb7d261640 1 -- 172.21.15.47:0/3960630526 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb78097fb0 con 0x7feb78097790 2024-09-15T21:56:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb7d261640 1 -- 172.21.15.47:0/3960630526 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7feb7806c280 con 0x7feb78096650 2024-09-15T21:56:36.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb7d261640 1 -- 172.21.15.47:0/3960630526 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7feb78096d20 con 0x7feb78097b70 2024-09-15T21:56:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f197f640 1 -- 172.21.15.47:0/2896567408 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097d20 msgr2=0x7fd2ec137300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f197f640 1 --2- 172.21.15.47:0/2896567408 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097d20 0x7fd2ec137300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f197f640 1 -- 172.21.15.47:0/2896567408 >> v1:172.21.15.47:6789/0 conn(0x7fd2ec0983c0 legacy=0x7fd2ec097180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f197f640 1 -- 172.21.15.47:0/2896567408 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec138850 con 0x7fd2ec097890 2024-09-15T21:56:36.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7fd2f2981640 1 --2- 172.21.15.47:0/2896567408 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097d20 0x7fd2ec137300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb767fc640 1 -- 172.21.15.47:0/3960630526 >> v2:172.21.15.47:3300/0 conn(0x7feb78096650 msgr2=0x7feb78137900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb767fc640 1 --2- 172.21.15.47:0/3960630526 >> v2:172.21.15.47:3300/0 conn(0x7feb78096650 0x7feb78137900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb767fc640 1 -- 172.21.15.47:0/3960630526 >> v1:172.21.15.47:6789/0 conn(0x7feb78097790 legacy=0x7feb78095a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.428+0000 7feb767fc640 1 -- 172.21.15.47:0/3960630526 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb78091820 con 0x7feb78097b70 2024-09-15T21:56:36.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.429+0000 7fd2f197f640 1 --2- 172.21.15.47:0/2896567408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec097890 0x7fd2ec134db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0097a0 tx=0x7fd2dc02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be4d59189718b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.429+0000 7feb767fc640 1 --2- 172.21.15.47:0/3960630526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097b70 0x7feb78096110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0097a0 tx=0x7feb6c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d10b2e2325daa608 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.429+0000 7fd2f117e640 1 -- 172.21.15.47:0/2896567408 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc037070 con 0x7fd2ec097890 2024-09-15T21:56:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.429+0000 7fd2f117e640 1 -- 172.21.15.47:0/2896567408 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2dc02adb0 con 0x7fd2ec097890 2024-09-15T21:56:36.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.429+0000 7fd2f117e640 1 -- 172.21.15.47:0/2896567408 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc032660 con 0x7fd2ec097890 2024-09-15T21:56:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb75ffb640 1 -- 172.21.15.47:0/3960630526 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c037070 con 0x7feb78097b70 2024-09-15T21:56:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb75ffb640 1 -- 172.21.15.47:0/3960630526 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c02adb0 con 0x7feb78097b70 2024-09-15T21:56:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb75ffb640 1 -- 172.21.15.47:0/3960630526 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c032660 con 0x7feb78097b70 2024-09-15T21:56:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7fd2f43e4640 1 -- 172.21.15.47:0/2896567408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec097890 msgr2=0x7fd2ec134db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7fd2f43e4640 1 --2- 172.21.15.47:0/2896567408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec097890 0x7fd2ec134db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0097a0 tx=0x7fd2dc02a0a0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb7d261640 1 -- 172.21.15.47:0/3960630526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097b70 msgr2=0x7feb78096110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb7d261640 1 --2- 172.21.15.47:0/3960630526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097b70 0x7feb78096110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0097a0 tx=0x7feb6c02a060 comp rx=0 tx=0).stop 2024-09-15T21:56:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7fd2f43e4640 1 -- 172.21.15.47:0/2896567408 shutdown_connections 2024-09-15T21:56:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7fd2f43e4640 1 --2- 172.21.15.47:0/2896567408 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097d20 0x7fd2ec137300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7fd2f43e4640 1 --2- 172.21.15.47:0/2896567408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec097890 0x7fd2ec134db0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7fd2f43e4640 1 -- 172.21.15.47:0/2896567408 >> 172.21.15.47:0/2896567408 conn(0x7fd2ec09fdf0 msgr2=0x7fd2ec0a0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb7d261640 1 -- 172.21.15.47:0/3960630526 shutdown_connections 2024-09-15T21:56:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb7d261640 1 --2- 172.21.15.47:0/3960630526 >> v2:172.21.15.47:3300/0 conn(0x7feb78096650 0x7feb78137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb7d261640 1 --2- 172.21.15.47:0/3960630526 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097b70 0x7feb78096110 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb7d261640 1 -- 172.21.15.47:0/3960630526 >> 172.21.15.47:0/3960630526 conn(0x7feb7809c310 msgr2=0x7feb7809c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7fd2f43e4640 1 -- 172.21.15.47:0/2896567408 shutdown_connections 2024-09-15T21:56:36.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7fd2f43e4640 1 -- 172.21.15.47:0/2896567408 wait complete. 2024-09-15T21:56:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb7d261640 1 -- 172.21.15.47:0/3960630526 shutdown_connections 2024-09-15T21:56:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.430+0000 7feb7d261640 1 -- 172.21.15.47:0/3960630526 wait complete. 2024-09-15T21:56:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.431+0000 7feb7d261640 1 Processor -- start 2024-09-15T21:56:36.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7fd2f43e4640 1 Processor -- start 2024-09-15T21:56:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb7d261640 1 -- start start 2024-09-15T21:56:36.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7fd2f43e4640 1 -- start start 2024-09-15T21:56:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb7d261640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb78096650 0x7feb78125150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb7d261640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097790 0x7feb78125690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb7d261640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb78127200 0x7feb78125bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb7d261640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb780970f0 con 0x7feb78096650 2024-09-15T21:56:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb7d261640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7feb78068620 con 0x7feb78097790 2024-09-15T21:56:36.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb7d261640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7feb78137e40 con 0x7feb78127200 2024-09-15T21:56:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7fd2f43e4640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097890 0x7fd2ec12dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb767fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097790 0x7feb78125690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7fd2f43e4640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2ec097d20 0x7fd2ec12e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb777fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb78127200 0x7feb78125bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7fd2f43e4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec0983c0 0x7fd2ec134680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7fd2f2180640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097890 0x7fd2ec12dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7fd2f2180640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097890 0x7fd2ec12dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46758/0 (socket says 172.21.15.47:46758) 2024-09-15T21:56:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f2180640 1 -- 172.21.15.47:0/531271217 learned_addr learned my addr 172.21.15.47:0/531271217 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec137bd0 con 0x7fd2ec0983c0 2024-09-15T21:56:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ec137a50 con 0x7fd2ec097d20 2024-09-15T21:56:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd2ec137d50 con 0x7fd2ec097890 2024-09-15T21:56:36.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f197f640 1 --2- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2ec097d20 0x7fd2ec12e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb767fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097790 0x7feb78125690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40156/0 (socket says 172.21.15.47:40156) 2024-09-15T21:56:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb767fc640 1 -- 172.21.15.47:0/820457685 learned_addr learned my addr 172.21.15.47:0/820457685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.432+0000 7feb76ffd640 1 --2- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb78096650 0x7feb78125150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7feb767fc640 1 -- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb78127200 msgr2=0x7feb78125bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7feb767fc640 1 --2- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb78127200 0x7feb78125bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7feb767fc640 1 -- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb78096650 msgr2=0x7feb78125150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7feb767fc640 1 --2- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb78096650 0x7feb78125150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7feb767fc640 1 -- 172.21.15.47:0/820457685 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb78126200 con 0x7feb78097790 2024-09-15T21:56:36.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f2981640 1 --2- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec0983c0 0x7fd2ec134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f2180640 1 -- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2ec097d20 msgr2=0x7fd2ec12e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f2180640 1 --2- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2ec097d20 0x7fd2ec12e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7feb767fc640 1 --2- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097790 0x7feb78125690 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0098c0 tx=0x7feb6c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:36.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f2180640 1 -- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec0983c0 msgr2=0x7fd2ec134680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f2180640 1 --2- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec0983c0 0x7fd2ec134680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f2180640 1 -- 172.21.15.47:0/531271217 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec1368e0 con 0x7fd2ec097890 2024-09-15T21:56:36.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f197f640 1 --2- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2ec097d20 0x7fd2ec12e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:36.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7fd2f2180640 1 --2- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097890 0x7fd2ec12dfd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c430 tx=0x7fd2e800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:36.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.433+0000 7feb5f7fe640 1 -- 172.21.15.47:0/820457685 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c042070 con 0x7feb78097790 2024-09-15T21:56:36.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.434+0000 7feb5f7fe640 1 -- 172.21.15.47:0/820457685 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c03e460 con 0x7feb78097790 2024-09-15T21:56:36.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.434+0000 7feb7d261640 1 -- 172.21.15.47:0/820457685 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb78126b70 con 0x7feb78097790 2024-09-15T21:56:36.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.434+0000 7feb5f7fe640 1 -- 172.21.15.47:0/820457685 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c037040 con 0x7feb78097790 2024-09-15T21:56:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.434+0000 7fd2e2ffd640 1 -- 172.21.15.47:0/531271217 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8011790 con 0x7fd2ec097890 2024-09-15T21:56:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.434+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ec12ea50 con 0x7fd2ec097890 2024-09-15T21:56:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.434+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ec134da0 con 0x7fd2ec097890 2024-09-15T21:56:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.434+0000 7fd2e2ffd640 1 -- 172.21.15.47:0/531271217 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e8017070 con 0x7fd2ec097890 2024-09-15T21:56:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.434+0000 7fd2e2ffd640 1 -- 172.21.15.47:0/531271217 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8011bb0 con 0x7fd2ec097890 2024-09-15T21:56:36.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.434+0000 7feb7d261640 1 -- 172.21.15.47:0/820457685 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb781c4d50 con 0x7feb78097790 2024-09-15T21:56:36.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.435+0000 7feb5f7fe640 1 -- 172.21.15.47:0/820457685 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 9) ==== 55943+0+0 (secure 0 0 0) 0x7feb6c045070 con 0x7feb78097790 2024-09-15T21:56:36.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.435+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d4005290 con 0x7fd2ec097890 2024-09-15T21:56:36.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.436+0000 7feb7d261640 1 -- 172.21.15.47:0/820457685 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb78127be0 con 0x7feb78097790 2024-09-15T21:56:36.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.436+0000 7feb5f7fe640 1 -- 172.21.15.47:0/820457685 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7feb6c05e050 con 0x7feb78097790 2024-09-15T21:56:36.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.436+0000 7fd2e2ffd640 1 -- 172.21.15.47:0/531271217 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 9) ==== 55943+0+0 (secure 0 0 0) 0x7fd2e801c460 con 0x7fd2ec097890 2024-09-15T21:56:36.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.437+0000 7fd2e2ffd640 1 -- 172.21.15.47:0/531271217 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7fd2e805d6a0 con 0x7fd2ec097890 2024-09-15T21:56:36.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.440+0000 7feb5f7fe640 1 -- 172.21.15.47:0/820457685 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7feb6c07bca0 con 0x7feb78097790 2024-09-15T21:56:36.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.444+0000 7fd2e2ffd640 1 -- 172.21.15.47:0/531271217 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fd2e803c8b0 con 0x7fd2ec097890 2024-09-15T21:56:36.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.494+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fee00075130 con 0x7fee0009c000 2024-09-15T21:56:36.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.494+0000 7fede77fe640 1 -- 172.21.15.47:0/2146310303 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fedf4044ad0 con 0x7fee0009c000 2024-09-15T21:56:36.496 INFO:teuthology.orchestra.run.smithi047.stdout:38654705685 2024-09-15T21:56:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.496+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 >> v1:172.21.15.47:6800/1790418198 conn(0x7fedec047f40 legacy=0x7fedec04a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.496+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee0009c000 msgr2=0x7fee00135bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.496+0000 7fee0500d640 1 --2- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee0009c000 0x7fee00135bd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fedf400b170 tx=0x7fedf400b630 comp rx=0 tx=0).stop 2024-09-15T21:56:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.496+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 shutdown_connections 2024-09-15T21:56:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.496+0000 7fee0500d640 1 --2- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee0012c130 0x7fee00136110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.496+0000 7fee0500d640 1 --2- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fee0009c000 0x7fee00135bd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.496+0000 7fee0500d640 1 --2- 172.21.15.47:0/2146310303 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fee00095060 0x7fee00135690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.496+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 >> 172.21.15.47:0/2146310303 conn(0x7fee0009f2a0 msgr2=0x7fee00096170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.496+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 shutdown_connections 2024-09-15T21:56:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.496+0000 7fee0500d640 1 -- 172.21.15.47:0/2146310303 wait complete. 2024-09-15T21:56:36.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705684 got 38654705685 for osd.3 2024-09-15T21:56:36.509 DEBUG:teuthology.parallel:result is None 2024-09-15T21:56:36.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.552+0000 7fb524153640 1 -- 172.21.15.47:0/704660753 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb51c09d4d0 con 0x7fb51c0983c0 2024-09-15T21:56:36.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.552+0000 7fb5127fc640 1 -- 172.21.15.47:0/704660753 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb51c09d4d0 con 0x7fb51c0983c0 2024-09-15T21:56:36.554 INFO:teuthology.orchestra.run.smithi047.stdout:38654705685 2024-09-15T21:56:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.554+0000 7fb524153640 1 -- 172.21.15.47:0/704660753 >> v1:172.21.15.47:6800/1790418198 conn(0x7fb500047f90 legacy=0x7fb50004a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.554+0000 7fb524153640 1 -- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb51c0983c0 msgr2=0x7fb51c070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.554+0000 7fb524153640 1 --2- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb51c0983c0 0x7fb51c070b80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb50c035040 tx=0x7fb50c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.554+0000 7fb524153640 1 -- 172.21.15.47:0/704660753 shutdown_connections 2024-09-15T21:56:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.554+0000 7fb524153640 1 --2- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51c133da0 0x7fb51c0710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.554+0000 7fb524153640 1 --2- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb51c0983c0 0x7fb51c070b80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.554+0000 7fb524153640 1 --2- 172.21.15.47:0/704660753 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb51c0976c0 0x7fb51c070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.554+0000 7fb524153640 1 -- 172.21.15.47:0/704660753 >> 172.21.15.47:0/704660753 conn(0x7fb51c09fdf0 msgr2=0x7fb51c0913f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:36.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.554+0000 7fb524153640 1 -- 172.21.15.47:0/704660753 shutdown_connections 2024-09-15T21:56:36.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.554+0000 7fb524153640 1 -- 172.21.15.47:0/704660753 wait complete. 2024-09-15T21:56:36.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705684 got 38654705685 for osd.2 2024-09-15T21:56:36.566 DEBUG:teuthology.parallel:result is None 2024-09-15T21:56:36.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.592+0000 7feb7d261640 1 -- 172.21.15.47:0/820457685 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7feb78095a00 con 0x7feb78097790 2024-09-15T21:56:36.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.592+0000 7feb5f7fe640 1 -- 172.21.15.47:0/820457685 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feb6c07c020 con 0x7feb78097790 2024-09-15T21:56:36.594 INFO:teuthology.orchestra.run.smithi047.stdout:38654705684 2024-09-15T21:56:36.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.594+0000 7feb7d261640 1 -- 172.21.15.47:0/820457685 >> v1:172.21.15.47:6800/1790418198 conn(0x7feb64047f90 legacy=0x7feb6404a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.594+0000 7feb7d261640 1 -- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097790 msgr2=0x7feb78125690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.594+0000 7feb7d261640 1 --2- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097790 0x7feb78125690 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0098c0 tx=0x7feb6c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.594+0000 7feb7d261640 1 -- 172.21.15.47:0/820457685 shutdown_connections 2024-09-15T21:56:36.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.594+0000 7feb7d261640 1 --2- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7feb78127200 0x7feb78125bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.594+0000 7feb7d261640 1 --2- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7feb78097790 0x7feb78125690 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.594+0000 7feb7d261640 1 --2- 172.21.15.47:0/820457685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb78096650 0x7feb78125150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.594+0000 7feb7d261640 1 -- 172.21.15.47:0/820457685 >> 172.21.15.47:0/820457685 conn(0x7feb7809c310 msgr2=0x7feb78094700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:36.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.594+0000 7feb7d261640 1 -- 172.21.15.47:0/820457685 shutdown_connections 2024-09-15T21:56:36.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.594+0000 7feb7d261640 1 -- 172.21.15.47:0/820457685 wait complete. 2024-09-15T21:56:36.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.595+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd2d4005580 con 0x7fd2ec097890 2024-09-15T21:56:36.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.595+0000 7fd2e2ffd640 1 -- 172.21.15.47:0/531271217 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd2e803cb10 con 0x7fd2ec097890 2024-09-15T21:56:36.597 INFO:teuthology.orchestra.run.smithi047.stdout:38654705684 2024-09-15T21:56:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.597+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 >> v1:172.21.15.47:6800/1790418198 conn(0x7fd2d0047f70 legacy=0x7fd2d004a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.597+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097890 msgr2=0x7fd2ec12dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.597+0000 7fd2f43e4640 1 --2- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097890 0x7fd2ec12dfd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c430 tx=0x7fd2e800c8f0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.597+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 shutdown_connections 2024-09-15T21:56:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.597+0000 7fd2f43e4640 1 --2- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2ec0983c0 0x7fd2ec134680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.597+0000 7fd2f43e4640 1 --2- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd2ec097d20 0x7fd2ec12e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.597+0000 7fd2f43e4640 1 --2- 172.21.15.47:0/531271217 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd2ec097890 0x7fd2ec12dfd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.597+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 >> 172.21.15.47:0/531271217 conn(0x7fd2ec09fdf0 msgr2=0x7fd2ec130ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:36.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.597+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 shutdown_connections 2024-09-15T21:56:36.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.597+0000 7fd2f43e4640 1 -- 172.21.15.47:0/531271217 wait complete. 2024-09-15T21:56:36.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705684 for osd.0 2024-09-15T21:56:36.607 DEBUG:teuthology.parallel:result is None 2024-09-15T21:56:36.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705683 got 38654705684 for osd.1 2024-09-15T21:56:36.610 DEBUG:teuthology.parallel:result is None 2024-09-15T21:56:36.610 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:56:36.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:56:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.751+0000 7ff20d395640 1 Processor -- start 2024-09-15T21:56:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.751+0000 7ff20d395640 1 -- start start 2024-09-15T21:56:36.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.751+0000 7ff20d395640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff20812fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.751+0000 7ff20d395640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff20812c950 0x7ff208137a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.751+0000 7ff20d395640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff20809c2c0 con 0x7ff208130270 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.751+0000 7ff20d395640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7ff208130690 con 0x7ff20812f740 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff20d395640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7ff20806d080 con 0x7ff20812c950 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff206ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff20812fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff2077fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff20812c950 0x7ff208137a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff2067fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff208130270 0x7ff20812c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36972/0 (socket says 172.21.15.47:36972) 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff2067fc640 1 -- 172.21.15.47:0/4146318674 learned_addr learned my addr 172.21.15.47:0/4146318674 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff206ffd640 1 -- 172.21.15.47:0/4146318674 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff20812c950 msgr2=0x7ff208137a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff206ffd640 1 --2- 172.21.15.47:0/4146318674 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff20812c950 0x7ff208137a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff206ffd640 1 -- 172.21.15.47:0/4146318674 >> v1:172.21.15.47:6789/0 conn(0x7ff208130270 legacy=0x7ff20812c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff205ffb640 1 -- 172.21.15.47:0/4146318674 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1888986525 0 0) 0x7ff1fc001110 con 0x7ff208130270 2024-09-15T21:56:36.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff206ffd640 1 -- 172.21.15.47:0/4146318674 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20806d200 con 0x7ff20812f740 2024-09-15T21:56:36.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.752+0000 7ff2077fe640 1 --2- 172.21.15.47:0/4146318674 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff20812c950 0x7ff208137a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:36.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff206ffd640 1 --2- 172.21.15.47:0/4146318674 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff20812fd30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80097d0 tx=0x7ff1f802a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db182d0e340ee2eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:36.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff205ffb640 1 -- 172.21.15.47:0/4146318674 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8037070 con 0x7ff20812f740 2024-09-15T21:56:36.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff205ffb640 1 -- 172.21.15.47:0/4146318674 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f802ad90 con 0x7ff20812f740 2024-09-15T21:56:36.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff205ffb640 1 -- 172.21.15.47:0/4146318674 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8032660 con 0x7ff20812f740 2024-09-15T21:56:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff20d395640 1 -- 172.21.15.47:0/4146318674 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 msgr2=0x7ff20812fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff20d395640 1 --2- 172.21.15.47:0/4146318674 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff20812fd30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff1f80097d0 tx=0x7ff1f802a040 comp rx=0 tx=0).stop 2024-09-15T21:56:36.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff20d395640 1 -- 172.21.15.47:0/4146318674 shutdown_connections 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff20d395640 1 --2- 172.21.15.47:0/4146318674 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff20812c950 0x7ff208137a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff20d395640 1 --2- 172.21.15.47:0/4146318674 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff20812fd30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff20d395640 1 -- 172.21.15.47:0/4146318674 >> 172.21.15.47:0/4146318674 conn(0x7ff20809edd0 msgr2=0x7ff20809f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.753+0000 7ff20d395640 1 -- 172.21.15.47:0/4146318674 shutdown_connections 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.754+0000 7ff20d395640 1 -- 172.21.15.47:0/4146318674 wait complete. 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.754+0000 7ff20d395640 1 Processor -- start 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.754+0000 7ff20d395640 1 -- start start 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff20d395640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff20812c950 0x7ff208131520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff20d395640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff208131a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff20d395640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff208130270 0x7ff208131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff20d395640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff208137fc0 con 0x7ff20812c950 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff20d395640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7ff20812d030 con 0x7ff20812f740 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff20d395640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7ff208138140 con 0x7ff208130270 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff206ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff20812c950 0x7ff208131520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff2067fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff208131a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff2077fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff208130270 0x7ff208131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff2067fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff208131a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40184/0 (socket says 172.21.15.47:40184) 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff206ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff20812c950 0x7ff208131520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47966/0 (socket says 172.21.15.47:47966) 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff2067fc640 1 -- 172.21.15.47:0/2699329207 learned_addr learned my addr 172.21.15.47:0/2699329207 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff206ffd640 1 -- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff208130270 msgr2=0x7ff208131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff206ffd640 1 --2- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff208130270 0x7ff208131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff206ffd640 1 -- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 msgr2=0x7ff208131a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff206ffd640 1 --2- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff208131a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff206ffd640 1 -- 172.21.15.47:0/2699329207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff208133880 con 0x7ff20812c950 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff2077fe640 1 --2- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff208130270 0x7ff208131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.755+0000 7ff2067fc640 1 --2- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff208131a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.756+0000 7ff206ffd640 1 --2- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff20812c950 0x7ff208131520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8031000 tx=0x7ff1f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.756+0000 7ff1ef7fe640 1 -- 172.21.15.47:0/2699329207 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8041020 con 0x7ff20812c950 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.756+0000 7ff1ef7fe640 1 -- 172.21.15.47:0/2699329207 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f803b430 con 0x7ff20812c950 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.756+0000 7ff20d395640 1 -- 172.21.15.47:0/2699329207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff208133050 con 0x7ff20812c950 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.756+0000 7ff1ef7fe640 1 -- 172.21.15.47:0/2699329207 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8037040 con 0x7ff20812c950 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.756+0000 7ff20d395640 1 -- 172.21.15.47:0/2699329207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2081343f0 con 0x7ff20812c950 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.757+0000 7ff1ef7fe640 1 -- 172.21.15.47:0/2699329207 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) ==== 55943+0+0 (secure 0 0 0) 0x7ff1f8044070 con 0x7ff20812c950 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.757+0000 7ff20d395640 1 -- 172.21.15.47:0/2699329207 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff208134da0 con 0x7ff20812c950 2024-09-15T21:56:36.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.757+0000 7ff1ef7fe640 1 -- 172.21.15.47:0/2699329207 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7ff1f807ce50 con 0x7ff20812c950 2024-09-15T21:56:36.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.761+0000 7ff1ef7fe640 1 -- 172.21.15.47:0/2699329207 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7ff1f805c0c0 con 0x7ff20812c950 2024-09-15T21:56:36.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.905+0000 7ff20d395640 1 -- 172.21.15.47:0/2699329207 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff20812fd30 con 0x7ff1f4047f40 2024-09-15T21:56:36.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.906+0000 7ff1ef7fe640 1 -- 172.21.15.47:0/2699329207 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32036 (unknown 2965378022 0 1700907509) 0x7ff20812fd30 con 0x7ff1f4047f40 2024-09-15T21:56:36.908 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:36.908 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:56:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.907+0000 7ff20d395640 1 -- 172.21.15.47:0/2699329207 >> v1:172.21.15.47:6800/1790418198 conn(0x7ff1f4047f40 legacy=0x7ff1f404a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.907+0000 7ff20d395640 1 -- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff20812c950 msgr2=0x7ff208131520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.907+0000 7ff20d395640 1 --2- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff20812c950 0x7ff208131520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8031000 tx=0x7ff1f80046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.907+0000 7ff20d395640 1 -- 172.21.15.47:0/2699329207 shutdown_connections 2024-09-15T21:56:36.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.907+0000 7ff20d395640 1 --2- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff208130270 0x7ff208131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.907+0000 7ff20d395640 1 --2- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff20812f740 0x7ff208131a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.907+0000 7ff20d395640 1 --2- 172.21.15.47:0/2699329207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff20812c950 0x7ff208131520 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.908+0000 7ff20d395640 1 -- 172.21.15.47:0/2699329207 >> 172.21.15.47:0/2699329207 conn(0x7ff20809edd0 msgr2=0x7ff20806ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.908+0000 7ff20d395640 1 -- 172.21.15.47:0/2699329207 shutdown_connections 2024-09-15T21:56:36.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:36.908+0000 7ff20d395640 1 -- 172.21.15.47:0/2699329207 wait complete. 2024-09-15T21:56:36.919 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-15T21:56:35.672442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1207864,"kb_used_data":1800,"kb_used_omap":6,"kb_used_meta":1206009,"kb_avail":376279496,"statfs":{"total":386547056640,"available":385310203904,"internally_reserved":0,"allocated":1843200,"data_stored":1370368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1234953568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.997277"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.709734+0000","last_change":"2024-09-15T21:55:20.285996+0000","last_active":"2024-09-15T21:55:26.709734+0000","last_peered":"2024-09-15T21:55:26.709734+0000","last_clean":"2024-09-15T21:55:26.709734+0000","last_became_active":"2024-09-15T21:55:18.278669+0000","last_became_peered":"2024-09-15T21:55:18.278669+0000","last_unstale":"2024-09-15T21:55:26.709734+0000","last_undegraded":"2024-09-15T21:55:26.709734+0000","last_fullsized":"2024-09-15T21:55:26.709734+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:25.111524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.452539+0000","last_change":"2024-09-15T21:55:20.281027+0000","last_active":"2024-09-15T21:55:26.452539+0000","last_peered":"2024-09-15T21:55:26.452539+0000","last_clean":"2024-09-15T21:55:26.452539+0000","last_became_active":"2024-09-15T21:55:18.280627+0000","last_became_peered":"2024-09-15T21:55:18.280627+0000","last_unstale":"2024-09-15T21:55:26.452539+0000","last_undegraded":"2024-09-15T21:55:26.452539+0000","last_fullsized":"2024-09-15T21:55:26.452539+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:22.442431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.709695+0000","last_change":"2024-09-15T21:55:20.286607+0000","last_active":"2024-09-15T21:55:26.709695+0000","last_peered":"2024-09-15T21:55:26.709695+0000","last_clean":"2024-09-15T21:55:26.709695+0000","last_became_active":"2024-09-15T21:55:18.277825+0000","last_became_peered":"2024-09-15T21:55:18.277825+0000","last_unstale":"2024-09-15T21:55:26.709695+0000","last_undegraded":"2024-09-15T21:55:26.709695+0000","last_fullsized":"2024-09-15T21:55:26.709695+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:22.442431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.452534+0000","last_change":"2024-09-15T21:55:20.280811+0000","last_active":"2024-09-15T21:55:26.452534+0000","last_peered":"2024-09-15T21:55:26.452534+0000","last_clean":"2024-09-15T21:55:26.452534+0000","last_became_active":"2024-09-15T21:55:18.279130+0000","last_became_peered":"2024-09-15T21:55:18.279130+0000","last_unstale":"2024-09-15T21:55:26.452534+0000","last_undegraded":"2024-09-15T21:55:26.452534+0000","last_fullsized":"2024-09-15T21:55:26.452534+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:25.111524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.709691+0000","last_change":"2024-09-15T21:55:20.286526+0000","last_active":"2024-09-15T21:55:26.709691+0000","last_peered":"2024-09-15T21:55:26.709691+0000","last_clean":"2024-09-15T21:55:26.709691+0000","last_became_active":"2024-09-15T21:55:18.279213+0000","last_became_peered":"2024-09-15T21:55:18.279213+0000","last_unstale":"2024-09-15T21:55:26.709691+0000","last_undegraded":"2024-09-15T21:55:26.709691+0000","last_fullsized":"2024-09-15T21:55:26.709691+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:01.390413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.299728+0000","last_change":"2024-09-15T21:55:20.280869+0000","last_active":"2024-09-15T21:55:26.299728+0000","last_peered":"2024-09-15T21:55:26.299728+0000","last_clean":"2024-09-15T21:55:26.299728+0000","last_became_active":"2024-09-15T21:55:18.276731+0000","last_became_peered":"2024-09-15T21:55:18.276731+0000","last_unstale":"2024-09-15T21:55:26.299728+0000","last_undegraded":"2024-09-15T21:55:26.299728+0000","last_fullsized":"2024-09-15T21:55:26.299728+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:04.981724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.709687+0000","last_change":"2024-09-15T21:55:20.285977+0000","last_active":"2024-09-15T21:55:26.709687+0000","last_peered":"2024-09-15T21:55:26.709687+0000","last_clean":"2024-09-15T21:55:26.709687+0000","last_became_active":"2024-09-15T21:55:18.280367+0000","last_became_peered":"2024-09-15T21:55:18.280367+0000","last_unstale":"2024-09-15T21:55:26.709687+0000","last_undegraded":"2024-09-15T21:55:26.709687+0000","last_fullsized":"2024-09-15T21:55:26.709687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:03.529028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.709718+0000","last_change":"2024-09-15T21:55:20.286832+0000","last_active":"2024-09-15T21:55:26.709718+0000","last_peered":"2024-09-15T21:55:26.709718+0000","last_clean":"2024-09-15T21:55:26.709718+0000","last_became_active":"2024-09-15T21:55:18.278579+0000","last_became_peered":"2024-09-15T21:55:18.278579+0000","last_unstale":"2024-09-15T21:55:26.709718+0000","last_undegraded":"2024-09-15T21:55:26.709718+0000","last_fullsized":"2024-09-15T21:55:26.709718+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:15:44.779410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":83,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.452562+0000","last_change":"2024-09-15T21:55:12.324328+0000","last_active":"2024-09-15T21:55:26.452562+0000","last_peered":"2024-09-15T21:55:26.452562+0000","last_clean":"2024-09-15T21:55:26.452562+0000","last_became_active":"2024-09-15T21:55:12.323202+0000","last_became_peered":"2024-09-15T21:55:12.323202+0000","last_unstale":"2024-09-15T21:55:26.452562+0000","last_undegraded":"2024-09-15T21:55:26.452562+0000","last_fullsized":"2024-09-15T21:55:26.452562+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_clean_scrub_stamp":"2024-09-15T21:55:11.249936+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:04.912041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705685,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301680,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070160,"statfs":{"total":96636764160,"available":96327843840,"internally_reserved":0,"allocated":167936,"data_stored":48275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301676,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070164,"statfs":{"total":96636764160,"available":96327847936,"internally_reserved":0,"allocated":163840,"data_stored":48256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705684,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":753664,"data_stored":635213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705684,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":757760,"data_stored":638624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:56:36.920 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:56:37.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda5b06b640 1 Processor -- start 2024-09-15T21:56:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda5b06b640 1 -- start start 2024-09-15T21:56:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda5b06b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda54096b30 0x7fda54097180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda5b06b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda540976c0 0x7fda54097b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda5b06b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda5409b890 con 0x7fda54098060 2024-09-15T21:56:37.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda5b06b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fda54099470 con 0x7fda54096b30 2024-09-15T21:56:37.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda5b06b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fda5409c3f0 con 0x7fda540976c0 2024-09-15T21:56:37.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda58e07640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda54096b30 0x7fda54097180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda53fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda540976c0 0x7fda54097b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda59608640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fda54098060 0x7fda54135db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36980/0 (socket says 172.21.15.47:36980) 2024-09-15T21:56:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.020+0000 7fda59608640 1 -- 172.21.15.47:0/1917884926 learned_addr learned my addr 172.21.15.47:0/1917884926 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.021+0000 7fda537fe640 1 -- 172.21.15.47:0/1917884926 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109374963 0 0) 0x7fda5409b890 con 0x7fda54098060 2024-09-15T21:56:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.021+0000 7fda537fe640 1 -- 172.21.15.47:0/1917884926 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda44003520 con 0x7fda54098060 2024-09-15T21:56:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.021+0000 7fda58e07640 1 -- 172.21.15.47:0/1917884926 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda540976c0 msgr2=0x7fda54097b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:37.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.021+0000 7fda58e07640 1 --2- 172.21.15.47:0/1917884926 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda540976c0 0x7fda54097b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.021+0000 7fda58e07640 1 -- 172.21.15.47:0/1917884926 >> v1:172.21.15.47:6789/0 conn(0x7fda54098060 legacy=0x7fda54135db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.021+0000 7fda58e07640 1 -- 172.21.15.47:0/1917884926 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda54098ed0 con 0x7fda54096b30 2024-09-15T21:56:37.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.021+0000 7fda53fff640 1 --2- 172.21.15.47:0/1917884926 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda540976c0 0x7fda54097b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.021+0000 7fda58e07640 1 --2- 172.21.15.47:0/1917884926 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda54096b30 0x7fda54097180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda480097d0 tx=0x7fda4802a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e721192ebf77d228 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:37.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda537fe640 1 -- 172.21.15.47:0/1917884926 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48037070 con 0x7fda54096b30 2024-09-15T21:56:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda537fe640 1 -- 172.21.15.47:0/1917884926 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4802ad90 con 0x7fda54096b30 2024-09-15T21:56:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda537fe640 1 -- 172.21.15.47:0/1917884926 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48032660 con 0x7fda54096b30 2024-09-15T21:56:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda5b06b640 1 -- 172.21.15.47:0/1917884926 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda54096b30 msgr2=0x7fda54097180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda5b06b640 1 --2- 172.21.15.47:0/1917884926 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda54096b30 0x7fda54097180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda480097d0 tx=0x7fda4802a040 comp rx=0 tx=0).stop 2024-09-15T21:56:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda5b06b640 1 -- 172.21.15.47:0/1917884926 shutdown_connections 2024-09-15T21:56:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda5b06b640 1 --2- 172.21.15.47:0/1917884926 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda540976c0 0x7fda54097b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda5b06b640 1 --2- 172.21.15.47:0/1917884926 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda54096b30 0x7fda54097180 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda5b06b640 1 -- 172.21.15.47:0/1917884926 >> 172.21.15.47:0/1917884926 conn(0x7fda5409c8f0 msgr2=0x7fda5409ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda5b06b640 1 -- 172.21.15.47:0/1917884926 shutdown_connections 2024-09-15T21:56:37.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.022+0000 7fda5b06b640 1 -- 172.21.15.47:0/1917884926 wait complete. 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.023+0000 7fda5b06b640 1 Processor -- start 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.023+0000 7fda5b06b640 1 -- start start 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda5b06b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda54096b30 0x7fda54124cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda5b06b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda540976c0 0x7fda54125210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda5b06b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda54098060 0x7fda54125750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda5b06b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fda54137860 con 0x7fda54098060 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda5b06b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fda541376e0 con 0x7fda540976c0 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda5b06b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fda541379e0 con 0x7fda54096b30 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda53fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda540976c0 0x7fda54125210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda58e07640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda54096b30 0x7fda54124cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda59608640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda54098060 0x7fda54125750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda53fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda540976c0 0x7fda54125210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40202/0 (socket says 172.21.15.47:40202) 2024-09-15T21:56:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda58e07640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda54096b30 0x7fda54124cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46806/0 (socket says 172.21.15.47:46806) 2024-09-15T21:56:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda53fff640 1 -- 172.21.15.47:0/378840804 learned_addr learned my addr 172.21.15.47:0/378840804 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda58e07640 1 -- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda540976c0 msgr2=0x7fda54125210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda58e07640 1 --2- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda540976c0 0x7fda54125210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda58e07640 1 -- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda54098060 msgr2=0x7fda54125750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda58e07640 1 --2- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda54098060 0x7fda54125750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda58e07640 1 -- 172.21.15.47:0/378840804 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda54126e60 con 0x7fda54096b30 2024-09-15T21:56:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda53fff640 1 --2- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda540976c0 0x7fda54125210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.024+0000 7fda58e07640 1 --2- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda54096b30 0x7fda54124cd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda5409ba60 tx=0x7fda480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:37.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.025+0000 7fda517fa640 1 -- 172.21.15.47:0/378840804 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48037040 con 0x7fda54096b30 2024-09-15T21:56:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.025+0000 7fda517fa640 1 -- 172.21.15.47:0/378840804 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4802ab90 con 0x7fda54096b30 2024-09-15T21:56:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.025+0000 7fda5b06b640 1 -- 172.21.15.47:0/378840804 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda54099e40 con 0x7fda54096b30 2024-09-15T21:56:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.025+0000 7fda517fa640 1 -- 172.21.15.47:0/378840804 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4803d660 con 0x7fda54096b30 2024-09-15T21:56:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.025+0000 7fda5b06b640 1 -- 172.21.15.47:0/378840804 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fda54137150 con 0x7fda54096b30 2024-09-15T21:56:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.026+0000 7fda5b06b640 1 -- 172.21.15.47:0/378840804 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda54097180 con 0x7fda54096b30 2024-09-15T21:56:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.026+0000 7fda517fa640 1 -- 172.21.15.47:0/378840804 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 9) ==== 55943+0+0 (secure 0 0 0) 0x7fda48044020 con 0x7fda54096b30 2024-09-15T21:56:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.026+0000 7fda517fa640 1 -- 172.21.15.47:0/378840804 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7fda4807d590 con 0x7fda54096b30 2024-09-15T21:56:37.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.030+0000 7fda517fa640 1 -- 172.21.15.47:0/378840804 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fda4802ad40 con 0x7fda54096b30 2024-09-15T21:56:37.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.174+0000 7fda5b06b640 1 -- 172.21.15.47:0/378840804 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda541264f0 con 0x7fda38047f40 2024-09-15T21:56:37.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.175+0000 7fda517fa640 1 -- 172.21.15.47:0/378840804 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32036 (unknown 2965378022 0 1700907509) 0x7fda541264f0 con 0x7fda38047f40 2024-09-15T21:56:37.177 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:37.177 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:56:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.176+0000 7fda5b06b640 1 -- 172.21.15.47:0/378840804 >> v1:172.21.15.47:6800/1790418198 conn(0x7fda38047f40 legacy=0x7fda3804a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.176+0000 7fda5b06b640 1 -- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda54096b30 msgr2=0x7fda54124cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.176+0000 7fda5b06b640 1 --2- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda54096b30 0x7fda54124cd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda5409ba60 tx=0x7fda480046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.176+0000 7fda5b06b640 1 -- 172.21.15.47:0/378840804 shutdown_connections 2024-09-15T21:56:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.176+0000 7fda5b06b640 1 --2- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda54098060 0x7fda54125750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.176+0000 7fda5b06b640 1 --2- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fda540976c0 0x7fda54125210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.176+0000 7fda5b06b640 1 --2- 172.21.15.47:0/378840804 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fda54096b30 0x7fda54124cd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.177+0000 7fda5b06b640 1 -- 172.21.15.47:0/378840804 >> 172.21.15.47:0/378840804 conn(0x7fda5409c8f0 msgr2=0x7fda54094890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.177+0000 7fda5b06b640 1 -- 172.21.15.47:0/378840804 shutdown_connections 2024-09-15T21:56:37.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.177+0000 7fda5b06b640 1 -- 172.21.15.47:0/378840804 wait complete. 2024-09-15T21:56:37.189 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-15T21:56:35.672442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1207864,"kb_used_data":1800,"kb_used_omap":6,"kb_used_meta":1206009,"kb_avail":376279496,"statfs":{"total":386547056640,"available":385310203904,"internally_reserved":0,"allocated":1843200,"data_stored":1370368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1234953568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.997277"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.709734+0000","last_change":"2024-09-15T21:55:20.285996+0000","last_active":"2024-09-15T21:55:26.709734+0000","last_peered":"2024-09-15T21:55:26.709734+0000","last_clean":"2024-09-15T21:55:26.709734+0000","last_became_active":"2024-09-15T21:55:18.278669+0000","last_became_peered":"2024-09-15T21:55:18.278669+0000","last_unstale":"2024-09-15T21:55:26.709734+0000","last_undegraded":"2024-09-15T21:55:26.709734+0000","last_fullsized":"2024-09-15T21:55:26.709734+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:25.111524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.452539+0000","last_change":"2024-09-15T21:55:20.281027+0000","last_active":"2024-09-15T21:55:26.452539+0000","last_peered":"2024-09-15T21:55:26.452539+0000","last_clean":"2024-09-15T21:55:26.452539+0000","last_became_active":"2024-09-15T21:55:18.280627+0000","last_became_peered":"2024-09-15T21:55:18.280627+0000","last_unstale":"2024-09-15T21:55:26.452539+0000","last_undegraded":"2024-09-15T21:55:26.452539+0000","last_fullsized":"2024-09-15T21:55:26.452539+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:22.442431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.709695+0000","last_change":"2024-09-15T21:55:20.286607+0000","last_active":"2024-09-15T21:55:26.709695+0000","last_peered":"2024-09-15T21:55:26.709695+0000","last_clean":"2024-09-15T21:55:26.709695+0000","last_became_active":"2024-09-15T21:55:18.277825+0000","last_became_peered":"2024-09-15T21:55:18.277825+0000","last_unstale":"2024-09-15T21:55:26.709695+0000","last_undegraded":"2024-09-15T21:55:26.709695+0000","last_fullsized":"2024-09-15T21:55:26.709695+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:22.442431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.452534+0000","last_change":"2024-09-15T21:55:20.280811+0000","last_active":"2024-09-15T21:55:26.452534+0000","last_peered":"2024-09-15T21:55:26.452534+0000","last_clean":"2024-09-15T21:55:26.452534+0000","last_became_active":"2024-09-15T21:55:18.279130+0000","last_became_peered":"2024-09-15T21:55:18.279130+0000","last_unstale":"2024-09-15T21:55:26.452534+0000","last_undegraded":"2024-09-15T21:55:26.452534+0000","last_fullsized":"2024-09-15T21:55:26.452534+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:51:25.111524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.709691+0000","last_change":"2024-09-15T21:55:20.286526+0000","last_active":"2024-09-15T21:55:26.709691+0000","last_peered":"2024-09-15T21:55:26.709691+0000","last_clean":"2024-09-15T21:55:26.709691+0000","last_became_active":"2024-09-15T21:55:18.279213+0000","last_became_peered":"2024-09-15T21:55:18.279213+0000","last_unstale":"2024-09-15T21:55:26.709691+0000","last_undegraded":"2024-09-15T21:55:26.709691+0000","last_fullsized":"2024-09-15T21:55:26.709691+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:01.390413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.299728+0000","last_change":"2024-09-15T21:55:20.280869+0000","last_active":"2024-09-15T21:55:26.299728+0000","last_peered":"2024-09-15T21:55:26.299728+0000","last_clean":"2024-09-15T21:55:26.299728+0000","last_became_active":"2024-09-15T21:55:18.276731+0000","last_became_peered":"2024-09-15T21:55:18.276731+0000","last_unstale":"2024-09-15T21:55:26.299728+0000","last_undegraded":"2024-09-15T21:55:26.299728+0000","last_fullsized":"2024-09-15T21:55:26.299728+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:04.981724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.709687+0000","last_change":"2024-09-15T21:55:20.285977+0000","last_active":"2024-09-15T21:55:26.709687+0000","last_peered":"2024-09-15T21:55:26.709687+0000","last_clean":"2024-09-15T21:55:26.709687+0000","last_became_active":"2024-09-15T21:55:18.280367+0000","last_became_peered":"2024-09-15T21:55:18.280367+0000","last_unstale":"2024-09-15T21:55:26.709687+0000","last_undegraded":"2024-09-15T21:55:26.709687+0000","last_fullsized":"2024-09-15T21:55:26.709687+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:03.529028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.709718+0000","last_change":"2024-09-15T21:55:20.286832+0000","last_active":"2024-09-15T21:55:26.709718+0000","last_peered":"2024-09-15T21:55:26.709718+0000","last_clean":"2024-09-15T21:55:26.709718+0000","last_became_active":"2024-09-15T21:55:18.278579+0000","last_became_peered":"2024-09-15T21:55:18.278579+0000","last_unstale":"2024-09-15T21:55:26.709718+0000","last_undegraded":"2024-09-15T21:55:26.709718+0000","last_fullsized":"2024-09-15T21:55:26.709718+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:15:44.779410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":83,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T21:55:26.452562+0000","last_change":"2024-09-15T21:55:12.324328+0000","last_active":"2024-09-15T21:55:26.452562+0000","last_peered":"2024-09-15T21:55:26.452562+0000","last_clean":"2024-09-15T21:55:26.452562+0000","last_became_active":"2024-09-15T21:55:12.323202+0000","last_became_peered":"2024-09-15T21:55:12.323202+0000","last_unstale":"2024-09-15T21:55:26.452562+0000","last_undegraded":"2024-09-15T21:55:26.452562+0000","last_fullsized":"2024-09-15T21:55:26.452562+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_clean_scrub_stamp":"2024-09-15T21:55:11.249936+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:04.912041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705685,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301680,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070160,"statfs":{"total":96636764160,"available":96327843840,"internally_reserved":0,"allocated":167936,"data_stored":48275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705685,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301676,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070164,"statfs":{"total":96636764160,"available":96327847936,"internally_reserved":0,"allocated":163840,"data_stored":48256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705684,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302252,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069588,"statfs":{"total":96636764160,"available":96327258112,"internally_reserved":0,"allocated":753664,"data_stored":635213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705684,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":757760,"data_stored":638624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:56:37.189 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:56:37.189 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T21:56:37.190 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T21:56:37.190 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:56:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f4657153640 1 Processor -- start 2024-09-15T21:56:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f4657153640 1 -- start start 2024-09-15T21:56:37.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f4657153640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4650096a00 0x7f4650130ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f4657153640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 0x7f4650133920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f4657153640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4650097790 con 0x7f465012bfb0 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f4657153640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f465006d630 con 0x7f4650096a00 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f4657153640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f4650097cb0 con 0x7f4650131530 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f4654eef640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f465012bfb0 0x7f465012c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36982/0 (socket says 172.21.15.47:36982) 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f464ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4650096a00 0x7f4650130ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f4654eef640 1 -- 172.21.15.47:0/3252608708 learned_addr learned my addr 172.21.15.47:0/3252608708 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.288+0000 7f46556f0640 1 --2- 172.21.15.47:0/3252608708 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 0x7f4650133920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.289+0000 7f464f7fe640 1 -- 172.21.15.47:0/3252608708 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2823588726 0 0) 0x7f4650097790 con 0x7f465012bfb0 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.289+0000 7f464f7fe640 1 -- 172.21.15.47:0/3252608708 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f463c003520 con 0x7f465012bfb0 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.289+0000 7f464ffff640 1 -- 172.21.15.47:0/3252608708 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 msgr2=0x7f4650133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.289+0000 7f464ffff640 1 --2- 172.21.15.47:0/3252608708 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 0x7f4650133920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.289+0000 7f464ffff640 1 -- 172.21.15.47:0/3252608708 >> v1:172.21.15.47:6789/0 conn(0x7f465012bfb0 legacy=0x7f465012c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.289+0000 7f464ffff640 1 -- 172.21.15.47:0/3252608708 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465012caa0 con 0x7f4650096a00 2024-09-15T21:56:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.289+0000 7f464ffff640 1 --2- 172.21.15.47:0/3252608708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 0x7f4650130ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4640009d90 tx=0x7f464002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9122124e636e26f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f464f7fe640 1 -- 172.21.15.47:0/3252608708 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640007740 con 0x7f4650096a00 2024-09-15T21:56:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f464f7fe640 1 -- 172.21.15.47:0/3252608708 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46400078e0 con 0x7f4650096a00 2024-09-15T21:56:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f464f7fe640 1 -- 172.21.15.47:0/3252608708 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640007c90 con 0x7f4650096a00 2024-09-15T21:56:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f4657153640 1 -- 172.21.15.47:0/3252608708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 msgr2=0x7f4650130ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f4657153640 1 --2- 172.21.15.47:0/3252608708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 0x7f4650130ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4640009d90 tx=0x7f464002a530 comp rx=0 tx=0).stop 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f4657153640 1 -- 172.21.15.47:0/3252608708 shutdown_connections 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f4657153640 1 --2- 172.21.15.47:0/3252608708 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 0x7f4650133920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f4657153640 1 --2- 172.21.15.47:0/3252608708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 0x7f4650130ff0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f4657153640 1 -- 172.21.15.47:0/3252608708 >> 172.21.15.47:0/3252608708 conn(0x7f465009ddd0 msgr2=0x7f465009e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f4657153640 1 -- 172.21.15.47:0/3252608708 shutdown_connections 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.290+0000 7f4657153640 1 -- 172.21.15.47:0/3252608708 wait complete. 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.291+0000 7f4657153640 1 Processor -- start 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.291+0000 7f4657153640 1 -- start start 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.291+0000 7f4657153640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 0x7f465008fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.291+0000 7f4657153640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f465012bfb0 0x7f46500902e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f4657153640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 0x7f46500934f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f4657153640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f46500952e0 con 0x7f4650096a00 2024-09-15T21:56:37.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f4657153640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f4650095160 con 0x7f465012bfb0 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f4657153640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f4650095460 con 0x7f4650131530 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f4654eef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 0x7f465008fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f46556f0640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 0x7f46500934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f4654eef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 0x7f465008fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47992/0 (socket says 172.21.15.47:47992) 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f4654eef640 1 -- 172.21.15.47:0/1107875612 learned_addr learned my addr 172.21.15.47:0/1107875612 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f464ffff640 1 --2- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f465012bfb0 0x7f46500902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f46556f0640 1 -- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f465012bfb0 msgr2=0x7f46500902e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f46556f0640 1 --2- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f465012bfb0 0x7f46500902e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f46556f0640 1 -- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 msgr2=0x7f465008fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f46556f0640 1 --2- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 0x7f465008fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f46556f0640 1 -- 172.21.15.47:0/1107875612 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46500915a0 con 0x7f4650131530 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f4654eef640 1 --2- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 0x7f465008fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:37.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f46556f0640 1 --2- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 0x7f46500934f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4650073d70 tx=0x7f464800eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f464d7fa640 1 -- 172.21.15.47:0/1107875612 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648016070 con 0x7f4650131530 2024-09-15T21:56:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f464d7fa640 1 -- 172.21.15.47:0/1107875612 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f464800c950 con 0x7f4650131530 2024-09-15T21:56:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f464d7fa640 1 -- 172.21.15.47:0/1107875612 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464800b040 con 0x7f4650131530 2024-09-15T21:56:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f4657153640 1 -- 172.21.15.47:0/1107875612 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4650090820 con 0x7f4650131530 2024-09-15T21:56:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.292+0000 7f4657153640 1 -- 172.21.15.47:0/1107875612 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f46500928d0 con 0x7f4650131530 2024-09-15T21:56:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.293+0000 7f464d7fa640 1 -- 172.21.15.47:0/1107875612 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 9) ==== 55943+0+0 (secure 0 0 0) 0x7f4648019070 con 0x7f4650131530 2024-09-15T21:56:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.293+0000 7f4657153640 1 -- 172.21.15.47:0/1107875612 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4650131010 con 0x7f4650131530 2024-09-15T21:56:37.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.294+0000 7f464d7fa640 1 -- 172.21.15.47:0/1107875612 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7f464805cf90 con 0x7f4650131530 2024-09-15T21:56:37.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.297+0000 7f464d7fa640 1 -- 172.21.15.47:0/1107875612 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f4650131010 con 0x7f4650131530 2024-09-15T21:56:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.467+0000 7f4657153640 1 -- 172.21.15.47:0/1107875612 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f465012c390 con 0x7f4650131530 2024-09-15T21:56:37.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.467+0000 7f464d7fa640 1 -- 172.21.15.47:0/1107875612 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f464803c1a0 con 0x7f4650131530 2024-09-15T21:56:37.469 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:37.469 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T21:56:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.469+0000 7f4657153640 1 -- 172.21.15.47:0/1107875612 >> v1:172.21.15.47:6800/1790418198 conn(0x7f4630047f40 legacy=0x7f463004a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.469+0000 7f4657153640 1 -- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 msgr2=0x7f46500934f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.469+0000 7f4657153640 1 --2- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 0x7f46500934f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4650073d70 tx=0x7f464800eac0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.469+0000 7f4657153640 1 -- 172.21.15.47:0/1107875612 shutdown_connections 2024-09-15T21:56:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.469+0000 7f4657153640 1 --2- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4650131530 0x7f46500934f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.469+0000 7f4657153640 1 --2- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f465012bfb0 0x7f46500902e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.469+0000 7f4657153640 1 --2- 172.21.15.47:0/1107875612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4650096a00 0x7f465008fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.469+0000 7f4657153640 1 -- 172.21.15.47:0/1107875612 >> 172.21.15.47:0/1107875612 conn(0x7f465009ddd0 msgr2=0x7f4650098e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:37.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.469+0000 7f4657153640 1 -- 172.21.15.47:0/1107875612 shutdown_connections 2024-09-15T21:56:37.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.469+0000 7f4657153640 1 -- 172.21.15.47:0/1107875612 wait complete. 2024-09-15T21:56:37.481 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T21:56:37.482 INFO:teuthology.run_tasks:Running task ceph.restart... 2024-09-15T21:56:37.492 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T21:56:37.492 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:56:37.524 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T21:56:37.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-15T21:56:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177b272640 1 Processor -- start 2024-09-15T21:56:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177b272640 1 -- start start 2024-09-15T21:56:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177b272640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f177406df70 0x7f177406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177b272640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177412fde0 0x7f1774130270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177b272640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f177406ef40 con 0x7f177406e890 2024-09-15T21:56:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177b272640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1774099490 con 0x7f177406df70 2024-09-15T21:56:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177b272640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f177409c4c0 con 0x7f177412fde0 2024-09-15T21:56:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177900e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f177406df70 0x7f177406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177880d640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f177406e890 0x7f177412f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36988/0 (socket says 172.21.15.47:36988) 2024-09-15T21:56:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177880d640 1 -- 172.21.15.47:0/169430035 learned_addr learned my addr 172.21.15.47:0/169430035 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177900e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f177406df70 0x7f177406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47998/0 (socket says 172.21.15.47:47998) 2024-09-15T21:56:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.665+0000 7f177980f640 1 --2- 172.21.15.47:0/169430035 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177412fde0 0x7f1774130270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.666+0000 7f177900e640 1 -- 172.21.15.47:0/169430035 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177412fde0 msgr2=0x7f1774130270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.666+0000 7f177900e640 1 --2- 172.21.15.47:0/169430035 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177412fde0 0x7f1774130270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.666+0000 7f177900e640 1 -- 172.21.15.47:0/169430035 >> v1:172.21.15.47:6789/0 conn(0x7f177406e890 legacy=0x7f177412f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.666+0000 7f176bfff640 1 -- 172.21.15.47:0/169430035 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 37232194 0 0) 0x7f177406ef40 con 0x7f177406e890 2024-09-15T21:56:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.666+0000 7f177900e640 1 -- 172.21.15.47:0/169430035 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1774099d20 con 0x7f177406df70 2024-09-15T21:56:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.666+0000 7f177980f640 1 --2- 172.21.15.47:0/169430035 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177412fde0 0x7f1774130270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.666+0000 7f177900e640 1 --2- 172.21.15.47:0/169430035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 0x7f177406e350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f17700097d0 tx=0x7f177002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d9ff12068d1f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.666+0000 7f176bfff640 1 -- 172.21.15.47:0/169430035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770037070 con 0x7f177406df70 2024-09-15T21:56:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.666+0000 7f176bfff640 1 -- 172.21.15.47:0/169430035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f177002ae40 con 0x7f177406df70 2024-09-15T21:56:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.667+0000 7f176bfff640 1 -- 172.21.15.47:0/169430035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770032660 con 0x7f177406df70 2024-09-15T21:56:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.667+0000 7f177b272640 1 -- 172.21.15.47:0/169430035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 msgr2=0x7f177406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.667+0000 7f177b272640 1 --2- 172.21.15.47:0/169430035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 0x7f177406e350 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f17700097d0 tx=0x7f177002a130 comp rx=0 tx=0).stop 2024-09-15T21:56:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.667+0000 7f177b272640 1 -- 172.21.15.47:0/169430035 shutdown_connections 2024-09-15T21:56:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.667+0000 7f177b272640 1 --2- 172.21.15.47:0/169430035 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177412fde0 0x7f1774130270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.667+0000 7f177b272640 1 --2- 172.21.15.47:0/169430035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 0x7f177406e350 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.667+0000 7f177b272640 1 -- 172.21.15.47:0/169430035 >> 172.21.15.47:0/169430035 conn(0x7f177409ec70 msgr2=0x7f177409f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.667+0000 7f177b272640 1 -- 172.21.15.47:0/169430035 shutdown_connections 2024-09-15T21:56:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.667+0000 7f177b272640 1 -- 172.21.15.47:0/169430035 wait complete. 2024-09-15T21:56:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.668+0000 7f177b272640 1 Processor -- start 2024-09-15T21:56:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.668+0000 7f177b272640 1 -- start start 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.668+0000 7f177b272640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 0x7f1774130ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.668+0000 7f177b272640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177406e890 0x7f1774131020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.668+0000 7f177b272640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f177412fde0 0x7f1774131560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177b272640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1774137b90 con 0x7f177406df70 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177b272640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f1774137a10 con 0x7f177406e890 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177b272640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f1774137d10 con 0x7f177412fde0 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177880d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177406e890 0x7f1774131020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177900e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 0x7f1774130ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177980f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f177412fde0 0x7f1774131560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177900e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 0x7f1774130ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48012/0 (socket says 172.21.15.47:48012) 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177900e640 1 -- 172.21.15.47:0/1448435022 learned_addr learned my addr 172.21.15.47:0/1448435022 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177900e640 1 -- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f177412fde0 msgr2=0x7f1774131560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177900e640 1 --2- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f177412fde0 0x7f1774131560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177900e640 1 -- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177406e890 msgr2=0x7f1774131020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177900e640 1 --2- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177406e890 0x7f1774131020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177900e640 1 -- 172.21.15.47:0/1448435022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1774133180 con 0x7f177406df70 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177880d640 1 --2- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177406e890 0x7f1774131020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177900e640 1 --2- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 0x7f1774130ae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1770035040 tx=0x7f17700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f1769ffb640 1 -- 172.21.15.47:0/1448435022 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770041070 con 0x7f177406df70 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f1769ffb640 1 -- 172.21.15.47:0/1448435022 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f177003c460 con 0x7f177406df70 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f177b272640 1 -- 172.21.15.47:0/1448435022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1774132580 con 0x7f177406df70 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.669+0000 7f1769ffb640 1 -- 172.21.15.47:0/1448435022 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770037040 con 0x7f177406df70 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.670+0000 7f177b272640 1 -- 172.21.15.47:0/1448435022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1774131c80 con 0x7f177406df70 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.670+0000 7f1769ffb640 1 -- 172.21.15.47:0/1448435022 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) ==== 55943+0+0 (secure 0 0 0) 0x7f1770044070 con 0x7f177406df70 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.670+0000 7f177b272640 1 -- 172.21.15.47:0/1448435022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177406c160 con 0x7f177406df70 2024-09-15T21:56:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.671+0000 7f1769ffb640 1 -- 172.21.15.47:0/1448435022 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3023+0+0 (secure 0 0 0) 0x7f177007de70 con 0x7f177406df70 2024-09-15T21:56:37.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.674+0000 7f1769ffb640 1 -- 172.21.15.47:0/1448435022 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f177406c160 con 0x7f177406df70 2024-09-15T21:56:37.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.680+0000 7f1769ffb640 1 -- 172.21.15.47:0/1448435022 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f177003a070 con 0x7f177406df70 2024-09-15T21:56:37.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.837+0000 7f177b272640 1 -- 172.21.15.47:0/1448435022 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd down", "ids": ["0"]} v 0) -- 0x7f17741342f0 con 0x7f177406df70 2024-09-15T21:56:37.839 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-09-15T21:56:37.837+0000 7fc963470640 -1 mon.a@0(leader).osd e20 definitely_dead 0 2024-09-15T21:56:37.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.837+0000 7f1769ffb640 1 -- 172.21.15.47:0/1448435022 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{"prefix": "osd down", "ids": ["0"]}]=0 osd.0 is already down. v20) ==== 93+0+0 (secure 0 0 0) 0x7f17741342f0 con 0x7f177406df70 2024-09-15T21:56:37.839 INFO:teuthology.orchestra.run.smithi047.stderr:osd.0 is already down. 2024-09-15T21:56:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.839+0000 7f177b272640 1 -- 172.21.15.47:0/1448435022 >> v1:172.21.15.47:6800/1790418198 conn(0x7f1758047f40 legacy=0x7f175804a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.839+0000 7f177b272640 1 -- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 msgr2=0x7f1774130ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.839+0000 7f177b272640 1 --2- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 0x7f1774130ae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1770035040 tx=0x7f17700046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.839+0000 7f177b272640 1 -- 172.21.15.47:0/1448435022 shutdown_connections 2024-09-15T21:56:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.839+0000 7f177b272640 1 --2- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f177412fde0 0x7f1774131560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.839+0000 7f177b272640 1 --2- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f177406e890 0x7f1774131020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.839+0000 7f177b272640 1 --2- 172.21.15.47:0/1448435022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f177406df70 0x7f1774130ae0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.839+0000 7f177b272640 1 -- 172.21.15.47:0/1448435022 >> 172.21.15.47:0/1448435022 conn(0x7f177409ec70 msgr2=0x7f177412f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.839+0000 7f177b272640 1 -- 172.21.15.47:0/1448435022 shutdown_connections 2024-09-15T21:56:37.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:37.839+0000 7f177b272640 1 -- 172.21.15.47:0/1448435022 wait complete. 2024-09-15T21:56:37.851 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T21:56:37.851 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T21:56:37.854 INFO:tasks.ceph.osd.0:Started 2024-09-15T21:56:37.854 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T21:56:37.854 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:56:37.888 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T21:56:37.888 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-15T21:56:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe29d296640 1 Processor -- start 2024-09-15T21:56:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe29d296640 1 -- start start 2024-09-15T21:56:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe29d296640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29806df70 0x7fe29806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe29d296640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29812fde0 0x7fe298130240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe29d296640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe29806ef40 con 0x7fe29806e890 2024-09-15T21:56:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe29d296640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fe29806ed10 con 0x7fe29812fde0 2024-09-15T21:56:38.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe29d296640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fe298099490 con 0x7fe29806df70 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe2967fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe29806e890 0x7fe29812f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:37008/0 (socket says 172.21.15.47:37008) 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe296ffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29806df70 0x7fe29806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe2967fc640 1 -- 172.21.15.47:0/420575482 learned_addr learned my addr 172.21.15.47:0/420575482 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe2977fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29812fde0 0x7fe298130240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.029+0000 7fe295ffb640 1 -- 172.21.15.47:0/420575482 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 691168840 0 0) 0x7fe29806ef40 con 0x7fe29806e890 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.030+0000 7fe295ffb640 1 -- 172.21.15.47:0/420575482 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe280003520 con 0x7fe29806e890 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.030+0000 7fe296ffd640 1 -- 172.21.15.47:0/420575482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29812fde0 msgr2=0x7fe298130240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.030+0000 7fe296ffd640 1 --2- 172.21.15.47:0/420575482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29812fde0 0x7fe298130240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.030+0000 7fe296ffd640 1 -- 172.21.15.47:0/420575482 >> v1:172.21.15.47:6789/0 conn(0x7fe29806e890 legacy=0x7fe29812f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.030+0000 7fe296ffd640 1 -- 172.21.15.47:0/420575482 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe298099d20 con 0x7fe29806df70 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.030+0000 7fe2977fe640 1 --2- 172.21.15.47:0/420575482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29812fde0 0x7fe298130240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:38.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.030+0000 7fe296ffd640 1 --2- 172.21.15.47:0/420575482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29806df70 0x7fe29806e350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe2880097d0 tx=0x7fe28802a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42b23462f85318a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.030+0000 7fe295ffb640 1 -- 172.21.15.47:0/420575482 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288037070 con 0x7fe29806df70 2024-09-15T21:56:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.030+0000 7fe295ffb640 1 -- 172.21.15.47:0/420575482 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe28802ae20 con 0x7fe29806df70 2024-09-15T21:56:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.030+0000 7fe295ffb640 1 -- 172.21.15.47:0/420575482 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288032660 con 0x7fe29806df70 2024-09-15T21:56:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.031+0000 7fe29d296640 1 -- 172.21.15.47:0/420575482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29806df70 msgr2=0x7fe29806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.031+0000 7fe29d296640 1 --2- 172.21.15.47:0/420575482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29806df70 0x7fe29806e350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe2880097d0 tx=0x7fe28802a0d0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.031+0000 7fe29d296640 1 -- 172.21.15.47:0/420575482 shutdown_connections 2024-09-15T21:56:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.031+0000 7fe29d296640 1 --2- 172.21.15.47:0/420575482 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29812fde0 0x7fe298130240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.031+0000 7fe29d296640 1 --2- 172.21.15.47:0/420575482 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29806df70 0x7fe29806e350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.031+0000 7fe29d296640 1 -- 172.21.15.47:0/420575482 >> 172.21.15.47:0/420575482 conn(0x7fe29809ec70 msgr2=0x7fe29809f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:38.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.031+0000 7fe29d296640 1 -- 172.21.15.47:0/420575482 shutdown_connections 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.031+0000 7fe29d296640 1 -- 172.21.15.47:0/420575482 wait complete. 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.032+0000 7fe29d296640 1 Processor -- start 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.032+0000 7fe29d296640 1 -- start start 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.032+0000 7fe29d296640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29806df70 0x7fe2981c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.032+0000 7fe29d296640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe29806e890 0x7fe2981c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.032+0000 7fe29d296640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29812fde0 0x7fe2981c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.032+0000 7fe29d296640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe298137a10 con 0x7fe29806e890 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.032+0000 7fe29d296640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fe29809c4c0 con 0x7fe29806df70 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.032+0000 7fe29d296640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fe298137b90 con 0x7fe29812fde0 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2967fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe29806e890 0x7fe2981c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2977fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29812fde0 0x7fe2981c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2967fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe29806e890 0x7fe2981c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48034/0 (socket says 172.21.15.47:48034) 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2967fc640 1 -- 172.21.15.47:0/2255522405 learned_addr learned my addr 172.21.15.47:0/2255522405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe296ffd640 1 --2- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29806df70 0x7fe2981c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2977fe640 1 -- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29806df70 msgr2=0x7fe2981c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2977fe640 1 --2- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29806df70 0x7fe2981c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2977fe640 1 -- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe29806e890 msgr2=0x7fe2981c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2977fe640 1 --2- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe29806e890 0x7fe2981c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2977fe640 1 -- 172.21.15.47:0/2255522405 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29812a080 con 0x7fe29812fde0 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2967fc640 1 --2- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe29806e890 0x7fe2981c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe2977fe640 1 --2- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29812fde0 0x7fe2981c56e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe29000efc0 tx=0x7fe29000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe27f7fe640 1 -- 172.21.15.47:0/2255522405 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290018030 con 0x7fe29812fde0 2024-09-15T21:56:38.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe27f7fe640 1 -- 172.21.15.47:0/2255522405 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29000cd30 con 0x7fe29812fde0 2024-09-15T21:56:38.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe29d296640 1 -- 172.21.15.47:0/2255522405 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe298132de0 con 0x7fe29812fde0 2024-09-15T21:56:38.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe27f7fe640 1 -- 172.21.15.47:0/2255522405 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290003bb0 con 0x7fe29812fde0 2024-09-15T21:56:38.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.033+0000 7fe29d296640 1 -- 172.21.15.47:0/2255522405 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2981312f0 con 0x7fe29812fde0 2024-09-15T21:56:38.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.034+0000 7fe27f7fe640 1 -- 172.21.15.47:0/2255522405 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fe290005d10 con 0x7fe29812fde0 2024-09-15T21:56:38.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.034+0000 7fe29d296640 1 -- 172.21.15.47:0/2255522405 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29806c160 con 0x7fe29812fde0 2024-09-15T21:56:38.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.035+0000 7fe27f7fe640 1 -- 172.21.15.47:0/2255522405 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(20..20 src has 1..20) ==== 3023+0+0 (secure 0 0 0) 0x7fe29005e2c0 con 0x7fe29812fde0 2024-09-15T21:56:38.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.038+0000 7fe27f7fe640 1 -- 172.21.15.47:0/2255522405 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fe29806c160 con 0x7fe29812fde0 2024-09-15T21:56:38.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.195+0000 7fe29d296640 1 -- 172.21.15.47:0/2255522405 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd down", "ids": ["1"]} v 0) -- 0x7fe29806e350 con 0x7fe29812fde0 2024-09-15T21:56:38.197 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-09-15T21:56:38.196+0000 7fc963470640 -1 mon.a@0(leader).osd e20 definitely_dead 0 2024-09-15T21:56:38.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.687+0000 7fe27f7fe640 1 -- 172.21.15.47:0/2255522405 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["1"]}]=0 marked down osd.1. v21) ==== 89+0+0 (secure 0 0 0) 0x7fe290022070 con 0x7fe29812fde0 2024-09-15T21:56:38.689 INFO:teuthology.orchestra.run.smithi047.stderr:marked down osd.1. 2024-09-15T21:56:38.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.690+0000 7fe29d296640 1 -- 172.21.15.47:0/2255522405 >> v1:172.21.15.47:6800/1790418198 conn(0x7fe284047fd0 legacy=0x7fe28404a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:38.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.690+0000 7fe29d296640 1 -- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29812fde0 msgr2=0x7fe2981c56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:38.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.690+0000 7fe29d296640 1 --2- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29812fde0 0x7fe2981c56e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe29000efc0 tx=0x7fe29000c490 comp rx=0 tx=0).stop 2024-09-15T21:56:38.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.690+0000 7fe29d296640 1 -- 172.21.15.47:0/2255522405 shutdown_connections 2024-09-15T21:56:38.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.690+0000 7fe29d296640 1 --2- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe29812fde0 0x7fe2981c56e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.690+0000 7fe29d296640 1 --2- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe29806e890 0x7fe2981c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.690+0000 7fe29d296640 1 --2- 172.21.15.47:0/2255522405 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe29806df70 0x7fe2981c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.690+0000 7fe29d296640 1 -- 172.21.15.47:0/2255522405 >> 172.21.15.47:0/2255522405 conn(0x7fe29809ec70 msgr2=0x7fe29809b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:38.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.690+0000 7fe29d296640 1 -- 172.21.15.47:0/2255522405 shutdown_connections 2024-09-15T21:56:38.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.690+0000 7fe29d296640 1 -- 172.21.15.47:0/2255522405 wait complete. 2024-09-15T21:56:38.705 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T21:56:38.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T21:56:38.707 INFO:tasks.ceph.osd.1:Started 2024-09-15T21:56:38.708 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T21:56:38.708 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:56:38.748 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T21:56:38.749 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-15T21:56:38.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.889+0000 7f39c4d6c640 1 Processor -- start 2024-09-15T21:56:38.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.889+0000 7f39c4d6c640 1 -- start start 2024-09-15T21:56:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39c4d6c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39c012fbf0 0x7f39c012ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39c4d6c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c006e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39c4d6c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c006e6c0 0x7f39c01367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39c4d6c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f39c0075b40 con 0x7f39c012fbf0 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39c4d6c640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f39c0099b90 con 0x7f39c006dd00 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39c4d6c640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f39c009c550 con 0x7f39c006e6c0 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bdd9b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c006e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bed9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c006e6c0 0x7f39c01367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bdd9b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c006e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40266/0 (socket says 172.21.15.47:40266) 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39be59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39c012fbf0 0x7f39c012ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bed9d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c006e6c0 0x7f39c01367f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:46880/0 (socket says 172.21.15.47:46880) 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bdd9b640 1 -- 172.21.15.47:0/322866596 learned_addr learned my addr 172.21.15.47:0/322866596 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bdd9b640 1 -- 172.21.15.47:0/322866596 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c006e6c0 msgr2=0x7f39c01367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bdd9b640 1 --2- 172.21.15.47:0/322866596 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c006e6c0 0x7f39c01367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bdd9b640 1 -- 172.21.15.47:0/322866596 >> v2:172.21.15.47:3300/0 conn(0x7f39c012fbf0 msgr2=0x7f39c012ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:38.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bdd9b640 1 --2- 172.21.15.47:0/322866596 >> v2:172.21.15.47:3300/0 conn(0x7f39c012fbf0 0x7f39c012ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bdd9b640 1 -- 172.21.15.47:0/322866596 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c012f730 con 0x7f39c006dd00 2024-09-15T21:56:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39be59c640 1 --2- 172.21.15.47:0/322866596 >> v2:172.21.15.47:3300/0 conn(0x7f39c012fbf0 0x7f39c012ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.890+0000 7f39bed9d640 1 --2- 172.21.15.47:0/322866596 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c006e6c0 0x7f39c01367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.891+0000 7f39bdd9b640 1 --2- 172.21.15.47:0/322866596 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c006e180 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f39b40098a0 tx=0x7f39b402a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae451d89be3178f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.891+0000 7f39bd59a640 1 -- 172.21.15.47:0/322866596 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4037070 con 0x7f39c006dd00 2024-09-15T21:56:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.891+0000 7f39bd59a640 1 -- 172.21.15.47:0/322866596 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b4032440 con 0x7f39c006dd00 2024-09-15T21:56:38.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.891+0000 7f39bd59a640 1 -- 172.21.15.47:0/322866596 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b402ae60 con 0x7f39c006dd00 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.892+0000 7f39c4d6c640 1 -- 172.21.15.47:0/322866596 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 msgr2=0x7f39c006e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.892+0000 7f39c4d6c640 1 --2- 172.21.15.47:0/322866596 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c006e180 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f39b40098a0 tx=0x7f39b402a150 comp rx=0 tx=0).stop 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.892+0000 7f39c4d6c640 1 -- 172.21.15.47:0/322866596 shutdown_connections 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.892+0000 7f39c4d6c640 1 --2- 172.21.15.47:0/322866596 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c006e6c0 0x7f39c01367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.892+0000 7f39c4d6c640 1 --2- 172.21.15.47:0/322866596 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c006e180 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.892+0000 7f39c4d6c640 1 --2- 172.21.15.47:0/322866596 >> v2:172.21.15.47:3300/0 conn(0x7f39c012fbf0 0x7f39c012ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.892+0000 7f39c4d6c640 1 -- 172.21.15.47:0/322866596 >> 172.21.15.47:0/322866596 conn(0x7f39c009ece0 msgr2=0x7f39c009f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.892+0000 7f39c4d6c640 1 -- 172.21.15.47:0/322866596 shutdown_connections 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.892+0000 7f39c4d6c640 1 -- 172.21.15.47:0/322866596 wait complete. 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39c4d6c640 1 Processor -- start 2024-09-15T21:56:38.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39c4d6c640 1 -- start start 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39c4d6c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c0127aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39c4d6c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c006e6c0 0x7f39c0127fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39c4d6c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c012fbf0 0x7f39c0128520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39c4d6c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f39c00997c0 con 0x7f39c006e6c0 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39c4d6c640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f39c0136d30 con 0x7f39c006dd00 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39c4d6c640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f39c0136eb0 con 0x7f39c012fbf0 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39be59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c0127aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39bdd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c006e6c0 0x7f39c0127fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39be59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c0127aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:40282/0 (socket says 172.21.15.47:40282) 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39be59c640 1 -- 172.21.15.47:0/4176532723 learned_addr learned my addr 172.21.15.47:0/4176532723 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:38.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.893+0000 7f39bed9d640 1 --2- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c012fbf0 0x7f39c0128520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39be59c640 1 -- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c012fbf0 msgr2=0x7f39c0128520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39be59c640 1 --2- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c012fbf0 0x7f39c0128520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39be59c640 1 -- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c006e6c0 msgr2=0x7f39c0127fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39be59c640 1 --2- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c006e6c0 0x7f39c0127fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39be59c640 1 -- 172.21.15.47:0/4176532723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c012b1e0 con 0x7f39c006dd00 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39bdd9b640 1 --2- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c006e6c0 0x7f39c0127fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39be59c640 1 --2- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c0127aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f39b000ebe0 tx=0x7f39b000c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39aeffd640 1 -- 172.21.15.47:0/4176532723 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0017070 con 0x7f39c006dd00 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39aeffd640 1 -- 172.21.15.47:0/4176532723 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b000cc30 con 0x7f39c006dd00 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39c4d6c640 1 -- 172.21.15.47:0/4176532723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c012a870 con 0x7f39c006dd00 2024-09-15T21:56:38.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39aeffd640 1 -- 172.21.15.47:0/4176532723 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0013470 con 0x7f39c006dd00 2024-09-15T21:56:38.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.894+0000 7f39c4d6c640 1 -- 172.21.15.47:0/4176532723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c0129e70 con 0x7f39c006dd00 2024-09-15T21:56:38.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.895+0000 7f39aeffd640 1 -- 172.21.15.47:0/4176532723 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f39b001a070 con 0x7f39c006dd00 2024-09-15T21:56:38.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.895+0000 7f39c4d6c640 1 -- 172.21.15.47:0/4176532723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c006e180 con 0x7f39c006dd00 2024-09-15T21:56:38.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.896+0000 7f39aeffd640 1 -- 172.21.15.47:0/4176532723 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 3023+0+0 (secure 0 0 0) 0x7f39b005dc60 con 0x7f39c006dd00 2024-09-15T21:56:38.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:38.899+0000 7f39aeffd640 1 -- 172.21.15.47:0/4176532723 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f39b005cd20 con 0x7f39c006dd00 2024-09-15T21:56:39.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.055+0000 7f39c4d6c640 1 -- 172.21.15.47:0/4176532723 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd down", "ids": ["2"]} v 0) -- 0x7f39c006f420 con 0x7f39c006dd00 2024-09-15T21:56:39.057 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-09-15T21:56:39.056+0000 7fc963470640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-15T21:56:39.529 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-09-15T21:56:39.528+0000 7fa04a4a5640 -1 Falling back to public interface 2024-09-15T21:56:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.693+0000 7f39aeffd640 1 -- 172.21.15.47:0/4176532723 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["2"]}]=0 marked down osd.2. v22) ==== 89+0+0 (secure 0 0 0) 0x7f39b003ce00 con 0x7f39c006dd00 2024-09-15T21:56:39.695 INFO:teuthology.orchestra.run.smithi047.stderr:marked down osd.2. 2024-09-15T21:56:39.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.695+0000 7f39c4d6c640 1 -- 172.21.15.47:0/4176532723 >> v1:172.21.15.47:6800/1790418198 conn(0x7f39a0047fd0 legacy=0x7f39a004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:39.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.695+0000 7f39c4d6c640 1 -- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 msgr2=0x7f39c0127aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:39.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.695+0000 7f39c4d6c640 1 --2- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c0127aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f39b000ebe0 tx=0x7f39b000c520 comp rx=0 tx=0).stop 2024-09-15T21:56:39.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.695+0000 7f39c4d6c640 1 -- 172.21.15.47:0/4176532723 shutdown_connections 2024-09-15T21:56:39.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.695+0000 7f39c4d6c640 1 --2- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f39c012fbf0 0x7f39c0128520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:39.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.695+0000 7f39c4d6c640 1 --2- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39c006e6c0 0x7f39c0127fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.695+0000 7f39c4d6c640 1 --2- 172.21.15.47:0/4176532723 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f39c006dd00 0x7f39c0127aa0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.695+0000 7f39c4d6c640 1 -- 172.21.15.47:0/4176532723 >> 172.21.15.47:0/4176532723 conn(0x7f39c009ece0 msgr2=0x7f39c009b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.695+0000 7f39c4d6c640 1 -- 172.21.15.47:0/4176532723 shutdown_connections 2024-09-15T21:56:39.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.696+0000 7f39c4d6c640 1 -- 172.21.15.47:0/4176532723 wait complete. 2024-09-15T21:56:39.710 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T21:56:39.710 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T21:56:39.713 INFO:tasks.ceph.osd.2:Started 2024-09-15T21:56:39.713 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T21:56:39.713 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:56:39.753 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T21:56:39.753 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 3 2024-09-15T21:56:39.880 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-09-15T21:56:39.878+0000 7f2dee7d9640 -1 Falling back to public interface 2024-09-15T21:56:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faffa5c2640 1 Processor -- start 2024-09-15T21:56:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faffa5c2640 1 -- start start 2024-09-15T21:56:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faffa5c2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faff409a630 0x7faff409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faffa5c2640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff412faf0 0x7faff412ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:39.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faffa5c2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faff409b600 con 0x7faff409af50 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faffa5c2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7faff406d480 con 0x7faff409a630 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faffa5c2640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7faff406d140 con 0x7faff412faf0 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faff3fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faff409a630 0x7faff409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faff8b5f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff412faf0 0x7faff412ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faff37fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7faff409af50 0x7faff412f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53726/0 (socket says 172.21.15.47:53726) 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.895+0000 7faff37fe640 1 -- 172.21.15.47:0/398556939 learned_addr learned my addr 172.21.15.47:0/398556939 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.896+0000 7faff8b5f640 1 -- 172.21.15.47:0/398556939 >> v2:172.21.15.47:3300/0 conn(0x7faff409a630 msgr2=0x7faff409aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.896+0000 7faff8b5f640 1 --2- 172.21.15.47:0/398556939 >> v2:172.21.15.47:3300/0 conn(0x7faff409a630 0x7faff409aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.896+0000 7faff8b5f640 1 -- 172.21.15.47:0/398556939 >> v1:172.21.15.47:6789/0 conn(0x7faff409af50 legacy=0x7faff412f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.896+0000 7faff8b5f640 1 -- 172.21.15.47:0/398556939 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff409bc00 con 0x7faff412faf0 2024-09-15T21:56:39.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.896+0000 7faff2ffd640 1 -- 172.21.15.47:0/398556939 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3380018473 0 0) 0x7faff409b600 con 0x7faff409af50 2024-09-15T21:56:39.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.896+0000 7faff3fff640 1 --2- 172.21.15.47:0/398556939 >> v2:172.21.15.47:3300/0 conn(0x7faff409a630 0x7faff409aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:39.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.896+0000 7faff8b5f640 1 --2- 172.21.15.47:0/398556939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff412faf0 0x7faff412ff60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fafec009d90 tx=0x7fafec02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd977ca072938b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faff2ffd640 1 -- 172.21.15.47:0/398556939 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec007740 con 0x7faff412faf0 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faff2ffd640 1 -- 172.21.15.47:0/398556939 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec0078e0 con 0x7faff412faf0 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faff2ffd640 1 -- 172.21.15.47:0/398556939 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec007cb0 con 0x7faff412faf0 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faffa5c2640 1 -- 172.21.15.47:0/398556939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff412faf0 msgr2=0x7faff412ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faffa5c2640 1 --2- 172.21.15.47:0/398556939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff412faf0 0x7faff412ff60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fafec009d90 tx=0x7fafec02a530 comp rx=0 tx=0).stop 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faffa5c2640 1 -- 172.21.15.47:0/398556939 shutdown_connections 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faffa5c2640 1 --2- 172.21.15.47:0/398556939 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff412faf0 0x7faff412ff60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faffa5c2640 1 --2- 172.21.15.47:0/398556939 >> v2:172.21.15.47:3300/0 conn(0x7faff409a630 0x7faff409aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faffa5c2640 1 -- 172.21.15.47:0/398556939 >> 172.21.15.47:0/398556939 conn(0x7faff409eb90 msgr2=0x7faff409efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faffa5c2640 1 -- 172.21.15.47:0/398556939 shutdown_connections 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.897+0000 7faffa5c2640 1 -- 172.21.15.47:0/398556939 wait complete. 2024-09-15T21:56:39.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.898+0000 7faffa5c2640 1 Processor -- start 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.898+0000 7faffa5c2640 1 -- start start 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faffa5c2640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff409a630 0x7faff4130b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faffa5c2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff409af50 0x7faff4131040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faffa5c2640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faff412faf0 0x7faff4131580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faffa5c2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faff4137ad0 con 0x7faff409af50 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faffa5c2640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7faff4137950 con 0x7faff409a630 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faffa5c2640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7faff4137c50 con 0x7faff412faf0 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff3fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff409a630 0x7faff4130b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff3fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff409a630 0x7faff4130b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59434/0 (socket says 172.21.15.47:59434) 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff3fff640 1 -- 172.21.15.47:0/1086011062 learned_addr learned my addr 172.21.15.47:0/1086011062 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff8b5f640 1 --2- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faff412faf0 0x7faff4131580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:39.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff37fe640 1 --2- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff409af50 0x7faff4131040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff3fff640 1 -- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faff412faf0 msgr2=0x7faff4131580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff3fff640 1 --2- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faff412faf0 0x7faff4131580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff3fff640 1 -- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff409af50 msgr2=0x7faff4131040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff3fff640 1 --2- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff409af50 0x7faff4131040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff3fff640 1 -- 172.21.15.47:0/1086011062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff4132670 con 0x7faff409a630 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff3fff640 1 --2- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff409a630 0x7faff4130b00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fafe400ebe0 tx=0x7fafe400c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff0ff9640 1 -- 172.21.15.47:0/1086011062 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe4017070 con 0x7faff409a630 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff0ff9640 1 -- 172.21.15.47:0/1086011062 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe400cc30 con 0x7faff409a630 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faff0ff9640 1 -- 172.21.15.47:0/1086011062 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe4013470 con 0x7faff409a630 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faffa5c2640 1 -- 172.21.15.47:0/1086011062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff4132ea0 con 0x7faff409a630 2024-09-15T21:56:39.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.899+0000 7faffa5c2640 1 -- 172.21.15.47:0/1086011062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faff4133a10 con 0x7faff409a630 2024-09-15T21:56:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.900+0000 7faffa5c2640 1 -- 172.21.15.47:0/1086011062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff406d9b0 con 0x7faff409a630 2024-09-15T21:56:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.901+0000 7faff0ff9640 1 -- 172.21.15.47:0/1086011062 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fafe401a070 con 0x7faff409a630 2024-09-15T21:56:39.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.901+0000 7faff0ff9640 1 -- 172.21.15.47:0/1086011062 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 3023+0+0 (secure 0 0 0) 0x7fafe405d900 con 0x7faff409a630 2024-09-15T21:56:39.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:39.904+0000 7faff0ff9640 1 -- 172.21.15.47:0/1086011062 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7faff406d9b0 con 0x7faff409a630 2024-09-15T21:56:40.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.061+0000 7faffa5c2640 1 -- 172.21.15.47:0/1086011062 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd down", "ids": ["3"]} v 0) -- 0x7faff409aa10 con 0x7faff409a630 2024-09-15T21:56:40.062 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-09-15T21:56:40.061+0000 7fc963470640 -1 mon.a@0(leader).osd e22 definitely_dead 0 2024-09-15T21:56:40.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.693+0000 7faff0ff9640 1 -- 172.21.15.47:0/1086011062 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["3"]}]=0 marked down osd.3. v23) ==== 89+0+0 (secure 0 0 0) 0x7fafe405c9c0 con 0x7faff409a630 2024-09-15T21:56:40.695 INFO:teuthology.orchestra.run.smithi047.stderr:marked down osd.3. 2024-09-15T21:56:40.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.695+0000 7faffa5c2640 1 -- 172.21.15.47:0/1086011062 >> v1:172.21.15.47:6800/1790418198 conn(0x7fafe0047fd0 legacy=0x7fafe004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:40.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.695+0000 7faffa5c2640 1 -- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff409a630 msgr2=0x7faff4130b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:40.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.695+0000 7faffa5c2640 1 --2- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff409a630 0x7faff4130b00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fafe400ebe0 tx=0x7fafe400c520 comp rx=0 tx=0).stop 2024-09-15T21:56:40.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.696+0000 7faffa5c2640 1 -- 172.21.15.47:0/1086011062 shutdown_connections 2024-09-15T21:56:40.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.696+0000 7faffa5c2640 1 --2- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faff412faf0 0x7faff4131580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.696+0000 7faffa5c2640 1 --2- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff409af50 0x7faff4131040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.696+0000 7faffa5c2640 1 --2- 172.21.15.47:0/1086011062 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faff409a630 0x7faff4130b00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.696+0000 7faffa5c2640 1 -- 172.21.15.47:0/1086011062 >> 172.21.15.47:0/1086011062 conn(0x7faff409eb90 msgr2=0x7faff412ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:40.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.696+0000 7faffa5c2640 1 -- 172.21.15.47:0/1086011062 shutdown_connections 2024-09-15T21:56:40.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.696+0000 7faffa5c2640 1 -- 172.21.15.47:0/1086011062 wait complete. 2024-09-15T21:56:40.709 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T21:56:40.710 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T21:56:40.712 INFO:tasks.ceph.osd.3:Started 2024-09-15T21:56:40.712 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:56:40.712 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:56:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.814+0000 7f1f74d5e640 1 Processor -- start 2024-09-15T21:56:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.814+0000 7f1f74d5e640 1 -- start start 2024-09-15T21:56:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f74d5e640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f7012f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f74d5e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1f7012fce0 0x7f1f70098090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:40.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f74d5e640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f70098a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f74d5e640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1f7006f260 con 0x7f1f7012fce0 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f74d5e640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f1f70099600 con 0x7f1f700985d0 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f74d5e640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f1f7009f230 con 0x7f1f7012f3c0 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6ed9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f70098a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6e59c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f7012f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6dd9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1f7012fce0 0x7f1f70098090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6ed9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f70098a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59444/0 (socket says 172.21.15.47:59444) 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6ed9d640 1 -- 172.21.15.47:0/4049090449 learned_addr learned my addr 172.21.15.47:0/4049090449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6ed9d640 1 -- 172.21.15.47:0/4049090449 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 msgr2=0x7f1f7012f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6ed9d640 1 --2- 172.21.15.47:0/4049090449 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f7012f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6ed9d640 1 -- 172.21.15.47:0/4049090449 >> v2:172.21.15.47:3300/0 conn(0x7f1f7012fce0 msgr2=0x7f1f70098090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6ed9d640 1 --2- 172.21.15.47:0/4049090449 >> v2:172.21.15.47:3300/0 conn(0x7f1f7012fce0 0x7f1f70098090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6ed9d640 1 -- 172.21.15.47:0/4049090449 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f70130120 con 0x7f1f700985d0 2024-09-15T21:56:40.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.815+0000 7f1f6e59c640 1 --2- 172.21.15.47:0/4049090449 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f7012f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:40.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.816+0000 7f1f6ed9d640 1 --2- 172.21.15.47:0/4049090449 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f70098a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009e50 tx=0x7f1f6802a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=def8bfaa3dff1bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:40.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.816+0000 7f1f6d59a640 1 -- 172.21.15.47:0/4049090449 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68036070 con 0x7f1f700985d0 2024-09-15T21:56:40.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.816+0000 7f1f6d59a640 1 -- 172.21.15.47:0/4049090449 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f68031600 con 0x7f1f700985d0 2024-09-15T21:56:40.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.816+0000 7f1f6d59a640 1 -- 172.21.15.47:0/4049090449 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680319d0 con 0x7f1f700985d0 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.817+0000 7f1f74d5e640 1 -- 172.21.15.47:0/4049090449 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 msgr2=0x7f1f70098a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.817+0000 7f1f74d5e640 1 --2- 172.21.15.47:0/4049090449 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f70098a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009e50 tx=0x7f1f6802a560 comp rx=0 tx=0).stop 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.817+0000 7f1f74d5e640 1 -- 172.21.15.47:0/4049090449 shutdown_connections 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.817+0000 7f1f74d5e640 1 --2- 172.21.15.47:0/4049090449 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f70098a80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.817+0000 7f1f74d5e640 1 --2- 172.21.15.47:0/4049090449 >> v2:172.21.15.47:3300/0 conn(0x7f1f7012fce0 0x7f1f70098090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.817+0000 7f1f74d5e640 1 --2- 172.21.15.47:0/4049090449 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f7012f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.817+0000 7f1f74d5e640 1 -- 172.21.15.47:0/4049090449 >> 172.21.15.47:0/4049090449 conn(0x7f1f700a23b0 msgr2=0x7f1f7009e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.817+0000 7f1f74d5e640 1 -- 172.21.15.47:0/4049090449 shutdown_connections 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.817+0000 7f1f74d5e640 1 -- 172.21.15.47:0/4049090449 wait complete. 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.818+0000 7f1f74d5e640 1 Processor -- start 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.818+0000 7f1f74d5e640 1 -- start start 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.818+0000 7f1f74d5e640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f701270e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:40.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.818+0000 7f1f74d5e640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f70127620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.818+0000 7f1f74d5e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f7012fce0 0x7f1f70127b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.818+0000 7f1f74d5e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70136920 con 0x7f1f7012fce0 2024-09-15T21:56:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.818+0000 7f1f74d5e640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7006ee60 con 0x7f1f700985d0 2024-09-15T21:56:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.818+0000 7f1f74d5e640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f1f70136aa0 con 0x7f1f7012f3c0 2024-09-15T21:56:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.818+0000 7f1f6e59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f701270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6dd9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f70127620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:40.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f7012fce0 0x7f1f70127b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6dd9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f70127620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:55838/0 (socket says 172.21.15.47:55838) 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6dd9b640 1 -- 172.21.15.47:0/2193555919 learned_addr learned my addr 172.21.15.47:0/2193555919 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6ed9d640 1 -- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 msgr2=0x7f1f70127620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6ed9d640 1 --2- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f70127620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6ed9d640 1 -- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 msgr2=0x7f1f701270e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6ed9d640 1 --2- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f701270e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6ed9d640 1 -- 172.21.15.47:0/2193555919 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7009c160 con 0x7f1f7012fce0 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6dd9b640 1 --2- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f70127620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6e59c640 1 --2- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f701270e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f6ed9d640 1 --2- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f7012fce0 0x7f1f70127b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f68030000 tx=0x7f1f68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f5effd640 1 -- 172.21.15.47:0/2193555919 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68036050 con 0x7f1f7012fce0 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f5effd640 1 -- 172.21.15.47:0/2193555919 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f68037040 con 0x7f1f7012fce0 2024-09-15T21:56:40.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f74d5e640 1 -- 172.21.15.47:0/2193555919 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f70128ff0 con 0x7f1f7012fce0 2024-09-15T21:56:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f5effd640 1 -- 172.21.15.47:0/2193555919 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680079b0 con 0x7f1f7012fce0 2024-09-15T21:56:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.819+0000 7f1f74d5e640 1 -- 172.21.15.47:0/2193555919 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f701299d0 con 0x7f1f7012fce0 2024-09-15T21:56:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.820+0000 7f1f5effd640 1 -- 172.21.15.47:0/2193555919 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f1f68044020 con 0x7f1f7012fce0 2024-09-15T21:56:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.820+0000 7f1f74d5e640 1 -- 172.21.15.47:0/2193555919 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f7012f7a0 con 0x7f1f7012fce0 2024-09-15T21:56:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.821+0000 7f1f5effd640 1 -- 172.21.15.47:0/2193555919 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3023+0+0 (secure 0 0 0) 0x7f1f6807e100 con 0x7f1f7012fce0 2024-09-15T21:56:40.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.824+0000 7f1f5effd640 1 -- 172.21.15.47:0/2193555919 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f1f6807d1c0 con 0x7f1f7012fce0 2024-09-15T21:56:40.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.973+0000 7f1f74d5e640 1 -- 172.21.15.47:0/2193555919 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f70068e40 con 0x7f1f7012fce0 2024-09-15T21:56:40.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.974+0000 7f1f5effd640 1 -- 172.21.15.47:0/2193555919 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+8492 (secure 0 0 0) 0x7f1f6807cd00 con 0x7f1f7012fce0 2024-09-15T21:56:40.975 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:40.976 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:56:40.690629+0000","last_up_change":"2024-09-15T21:56:40.690629+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":32760}]},"public_addr":"172.21.15.47:6801/32760","cluster_addr":"172.21.15.47:6802/32760","heartbeat_back_addr":"172.21.15.47:6804/32760","heartbeat_front_addr":"172.21.15.47:6803/32760","state":["exists"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":20,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":32761}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":32761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":32761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":32761}]},"public_addr":"172.21.15.47:6805/32761","cluster_addr":"172.21.15.47:6806/32761","heartbeat_back_addr":"172.21.15.47:6808/32761","heartbeat_front_addr":"172.21.15.47:6807/32761","state":["exists"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":32772}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":32772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":32772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":32772}]},"public_addr":"172.21.15.47:6813/32772","cluster_addr":"172.21.15.47:6814/32772","heartbeat_back_addr":"172.21.15.47:6816/32772","heartbeat_front_addr":"172.21.15.47:6815/32772","state":["exists"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":32770}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":32770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":32770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":32770}]},"public_addr":"172.21.15.47:6809/32770","cluster_addr":"172.21.15.47:6810/32770","heartbeat_back_addr":"172.21.15.47:6812/32770","heartbeat_front_addr":"172.21.15.47:6811/32770","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-15T21:56:40.690629+0000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/3200219561":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/2444109955":"2024-09-16T21:56:33.661886+0000","172.21.15.47:6800/32267":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1739125583":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1981014788":"2024-09-16T21:56:33.661886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:56:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.975+0000 7f1f74d5e640 1 -- 172.21.15.47:0/2193555919 >> v1:172.21.15.47:6800/1790418198 conn(0x7f1f50047fd0 legacy=0x7f1f5004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.975+0000 7f1f74d5e640 1 -- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f7012fce0 msgr2=0x7f1f70127b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:40.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.975+0000 7f1f74d5e640 1 --2- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f7012fce0 0x7f1f70127b60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f68030000 tx=0x7f1f68004a40 comp rx=0 tx=0).stop 2024-09-15T21:56:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.975+0000 7f1f74d5e640 1 -- 172.21.15.47:0/2193555919 shutdown_connections 2024-09-15T21:56:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.975+0000 7f1f74d5e640 1 --2- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1f7012fce0 0x7f1f70127b60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.975+0000 7f1f74d5e640 1 --2- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1f7012f3c0 0x7f1f70127620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.975+0000 7f1f74d5e640 1 --2- 172.21.15.47:0/2193555919 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1f700985d0 0x7f1f701270e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.975+0000 7f1f74d5e640 1 -- 172.21.15.47:0/2193555919 >> 172.21.15.47:0/2193555919 conn(0x7f1f700a23b0 msgr2=0x7f1f7012e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.976+0000 7f1f74d5e640 1 -- 172.21.15.47:0/2193555919 shutdown_connections 2024-09-15T21:56:40.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:40.976+0000 7f1f74d5e640 1 -- 172.21.15.47:0/2193555919 wait complete. 2024-09-15T21:56:41.386 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-09-15T21:56:41.384+0000 7f221f80f640 -1 Falling back to public interface 2024-09-15T21:56:41.906 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-09-15T21:56:41.904+0000 7fcc05ddd640 -1 Falling back to public interface 2024-09-15T21:56:43.989 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:56:44.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.095+0000 7f008462b640 1 Processor -- start 2024-09-15T21:56:44.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.095+0000 7f008462b640 1 -- start start 2024-09-15T21:56:44.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.095+0000 7f008462b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 0x7f007c12f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:44.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f008462b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c0987a0 0x7f007c137930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:44.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f008462b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f007c09f230 con 0x7f007c12fce0 2024-09-15T21:56:44.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f008462b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f007c06f260 con 0x7f007c0987a0 2024-09-15T21:56:44.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f008462b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f007c099600 con 0x7f007c12f3c0 2024-09-15T21:56:44.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f00823c7640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 0x7f007c12f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:44.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f0082bc8640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c0987a0 0x7f007c137930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:44.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f0081bc6640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f007c12fce0 0x7f007c098090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53742/0 (socket says 172.21.15.47:53742) 2024-09-15T21:56:44.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f0081bc6640 1 -- 172.21.15.47:0/2857683796 learned_addr learned my addr 172.21.15.47:0/2857683796 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:44.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f00813c5640 1 -- 172.21.15.47:0/2857683796 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3306844401 0 0) 0x7f007c09f230 con 0x7f007c12fce0 2024-09-15T21:56:44.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f00823c7640 1 -- 172.21.15.47:0/2857683796 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c0987a0 msgr2=0x7f007c137930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:44.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f00823c7640 1 --2- 172.21.15.47:0/2857683796 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c0987a0 0x7f007c137930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:44.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f00823c7640 1 -- 172.21.15.47:0/2857683796 >> v1:172.21.15.47:6789/0 conn(0x7f007c12fce0 legacy=0x7f007c098090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:44.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f00823c7640 1 -- 172.21.15.47:0/2857683796 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c130120 con 0x7f007c12f3c0 2024-09-15T21:56:44.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.096+0000 7f0082bc8640 1 --2- 172.21.15.47:0/2857683796 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c0987a0 0x7f007c137930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.097+0000 7f00823c7640 1 --2- 172.21.15.47:0/2857683796 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 0x7f007c12f7a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f00780097d0 tx=0x7f007802a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5734a4652558d5e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.097+0000 7f00813c5640 1 -- 172.21.15.47:0/2857683796 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078037070 con 0x7f007c12f3c0 2024-09-15T21:56:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.097+0000 7f00813c5640 1 -- 172.21.15.47:0/2857683796 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f007802ad90 con 0x7f007c12f3c0 2024-09-15T21:56:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.097+0000 7f00813c5640 1 -- 172.21.15.47:0/2857683796 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078032660 con 0x7f007c12f3c0 2024-09-15T21:56:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.097+0000 7f008462b640 1 -- 172.21.15.47:0/2857683796 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 msgr2=0x7f007c12f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.097+0000 7f008462b640 1 --2- 172.21.15.47:0/2857683796 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 0x7f007c12f7a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f00780097d0 tx=0x7f007802a040 comp rx=0 tx=0).stop 2024-09-15T21:56:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.098+0000 7f008462b640 1 -- 172.21.15.47:0/2857683796 shutdown_connections 2024-09-15T21:56:44.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.098+0000 7f008462b640 1 --2- 172.21.15.47:0/2857683796 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c0987a0 0x7f007c137930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.098+0000 7f008462b640 1 --2- 172.21.15.47:0/2857683796 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 0x7f007c12f7a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.098+0000 7f008462b640 1 -- 172.21.15.47:0/2857683796 >> 172.21.15.47:0/2857683796 conn(0x7f007c0a23b0 msgr2=0x7f007c09e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.098+0000 7f008462b640 1 -- 172.21.15.47:0/2857683796 shutdown_connections 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.098+0000 7f008462b640 1 -- 172.21.15.47:0/2857683796 wait complete. 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.098+0000 7f008462b640 1 Processor -- start 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.098+0000 7f008462b640 1 -- start start 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f008462b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c0987a0 0x7f007c1c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f008462b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 0x7f007c1c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f008462b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c12fce0 0x7f007c08cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f008462b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f007c098c50 con 0x7f007c0987a0 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f008462b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f007c06ee60 con 0x7f007c12fce0 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f008462b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f007c098dd0 con 0x7f007c12f3c0 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f00823c7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c0987a0 0x7f007c1c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:44.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f0081bc6640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 0x7f007c1c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f00823c7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c0987a0 0x7f007c1c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56852/0 (socket says 172.21.15.47:56852) 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f00823c7640 1 -- 172.21.15.47:0/255267843 learned_addr learned my addr 172.21.15.47:0/255267843 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f0082bc8640 1 --2- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c12fce0 0x7f007c08cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f0081bc6640 1 -- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c12fce0 msgr2=0x7f007c08cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f0081bc6640 1 --2- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c12fce0 0x7f007c08cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f0081bc6640 1 -- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c0987a0 msgr2=0x7f007c1c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f0081bc6640 1 --2- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c0987a0 0x7f007c1c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f0081bc6640 1 -- 172.21.15.47:0/255267843 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c1c53e0 con 0x7f007c12f3c0 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f0082bc8640 1 --2- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c12fce0 0x7f007c08cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f00823c7640 1 --2- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c0987a0 0x7f007c1c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.099+0000 7f0081bc6640 1 --2- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 0x7f007c1c4ea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c3e0 tx=0x7f006c00c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.100+0000 7f0072ffd640 1 -- 172.21.15.47:0/255267843 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c0074e0 con 0x7f007c12f3c0 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.100+0000 7f0072ffd640 1 -- 172.21.15.47:0/255267843 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006c007680 con 0x7f007c12f3c0 2024-09-15T21:56:44.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.100+0000 7f008462b640 1 -- 172.21.15.47:0/255267843 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007c08f930 con 0x7f007c12f3c0 2024-09-15T21:56:44.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.100+0000 7f0072ffd640 1 -- 172.21.15.47:0/255267843 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c004a20 con 0x7f007c12f3c0 2024-09-15T21:56:44.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.100+0000 7f008462b640 1 -- 172.21.15.47:0/255267843 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f007c08d700 con 0x7f007c12f3c0 2024-09-15T21:56:44.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.101+0000 7f008462b640 1 -- 172.21.15.47:0/255267843 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f007c12f7a0 con 0x7f007c12f3c0 2024-09-15T21:56:44.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.101+0000 7f0072ffd640 1 -- 172.21.15.47:0/255267843 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f006c011070 con 0x7f007c12f3c0 2024-09-15T21:56:44.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.101+0000 7f0072ffd640 1 -- 172.21.15.47:0/255267843 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(23..23 src has 1..23) ==== 3023+0+0 (secure 0 0 0) 0x7f006c05d3b0 con 0x7f007c12f3c0 2024-09-15T21:56:44.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.105+0000 7f0072ffd640 1 -- 172.21.15.47:0/255267843 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f006c05c470 con 0x7f007c12f3c0 2024-09-15T21:56:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.252+0000 7f008462b640 1 -- 172.21.15.47:0/255267843 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f007c090530 con 0x7f007c12f3c0 2024-09-15T21:56:44.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.253+0000 7f0072ffd640 1 -- 172.21.15.47:0/255267843 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+8492 (secure 0 0 0) 0x7f006c03c5b0 con 0x7f007c12f3c0 2024-09-15T21:56:44.255 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:44.255 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:56:40.690629+0000","last_up_change":"2024-09-15T21:56:40.690629+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":32760}]},"public_addr":"172.21.15.47:6801/32760","cluster_addr":"172.21.15.47:6802/32760","heartbeat_back_addr":"172.21.15.47:6804/32760","heartbeat_front_addr":"172.21.15.47:6803/32760","state":["exists"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":20,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":32761}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":32761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":32761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":32761}]},"public_addr":"172.21.15.47:6805/32761","cluster_addr":"172.21.15.47:6806/32761","heartbeat_back_addr":"172.21.15.47:6808/32761","heartbeat_front_addr":"172.21.15.47:6807/32761","state":["exists"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":32772}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":32772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":32772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":32772}]},"public_addr":"172.21.15.47:6813/32772","cluster_addr":"172.21.15.47:6814/32772","heartbeat_back_addr":"172.21.15.47:6816/32772","heartbeat_front_addr":"172.21.15.47:6815/32772","state":["exists"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":32770}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":32770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":32770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":32770}]},"public_addr":"172.21.15.47:6809/32770","cluster_addr":"172.21.15.47:6810/32770","heartbeat_back_addr":"172.21.15.47:6812/32770","heartbeat_front_addr":"172.21.15.47:6811/32770","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-15T21:56:40.690629+0000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/3200219561":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/2444109955":"2024-09-16T21:56:33.661886+0000","172.21.15.47:6800/32267":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1739125583":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1981014788":"2024-09-16T21:56:33.661886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:56:44.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.255+0000 7f008462b640 1 -- 172.21.15.47:0/255267843 >> v1:172.21.15.47:6800/1790418198 conn(0x7f0060047fd0 legacy=0x7f006004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.255+0000 7f008462b640 1 -- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 msgr2=0x7f007c1c4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.255+0000 7f008462b640 1 --2- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 0x7f007c1c4ea0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c3e0 tx=0x7f006c00c7e0 comp rx=0 tx=0).stop 2024-09-15T21:56:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.255+0000 7f008462b640 1 -- 172.21.15.47:0/255267843 shutdown_connections 2024-09-15T21:56:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.255+0000 7f008462b640 1 --2- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f007c12fce0 0x7f007c08cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.255+0000 7f008462b640 1 --2- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f007c12f3c0 0x7f007c1c4ea0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.255+0000 7f008462b640 1 --2- 172.21.15.47:0/255267843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f007c0987a0 0x7f007c1c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.255+0000 7f008462b640 1 -- 172.21.15.47:0/255267843 >> 172.21.15.47:0/255267843 conn(0x7f007c0a23b0 msgr2=0x7f007c09e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.255+0000 7f008462b640 1 -- 172.21.15.47:0/255267843 shutdown_connections 2024-09-15T21:56:44.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:44.255+0000 7f008462b640 1 -- 172.21.15.47:0/255267843 wait complete. 2024-09-15T21:56:45.498 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-09-15T21:56:45.497+0000 7fa04a4a5640 -1 osd.0 19 log_to_monitors true 2024-09-15T21:56:45.580 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-09-15T21:56:45.579+0000 7f2dee7d9640 -1 osd.1 20 log_to_monitors true 2024-09-15T21:56:45.710 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-09-15T21:56:45.709+0000 7f2de2578640 -1 osd.1 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:56:45.712 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-09-15T21:56:45.711+0000 7fa03e244640 -1 osd.0 19 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:56:47.047 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-09-15T21:56:47.045+0000 7f221f80f640 -1 osd.2 21 log_to_monitors true 2024-09-15T21:56:47.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:56:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c95055640 1 Processor -- start 2024-09-15T21:56:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c95055640 1 -- start start 2024-09-15T21:56:47.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c95055640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c9012f3c0 0x7f4c9012f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c95055640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4c900987a0 0x7f4c90137930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c95055640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c9009f230 con 0x7f4c9012fce0 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c95055640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4c9006f260 con 0x7f4c900987a0 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c95055640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f4c90099600 con 0x7f4c9012f3c0 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c8f59e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4c900987a0 0x7f4c90137930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c8e59c640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4c9012fce0 0x7f4c90098090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53758/0 (socket says 172.21.15.47:53758) 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c8ed9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c9012f3c0 0x7f4c9012f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.368+0000 7f4c8e59c640 1 -- 172.21.15.47:0/2445418952 learned_addr learned my addr 172.21.15.47:0/2445418952 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.369+0000 7f4c8ed9d640 1 -- 172.21.15.47:0/2445418952 >> v2:172.21.15.47:3300/0 conn(0x7f4c900987a0 msgr2=0x7f4c90137930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.369+0000 7f4c8ed9d640 1 --2- 172.21.15.47:0/2445418952 >> v2:172.21.15.47:3300/0 conn(0x7f4c900987a0 0x7f4c90137930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.369+0000 7f4c8ed9d640 1 -- 172.21.15.47:0/2445418952 >> v1:172.21.15.47:6789/0 conn(0x7f4c9012fce0 legacy=0x7f4c90098090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.369+0000 7f4c8ed9d640 1 -- 172.21.15.47:0/2445418952 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c90130120 con 0x7f4c9012f3c0 2024-09-15T21:56:47.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.369+0000 7f4c8f59e640 1 --2- 172.21.15.47:0/2445418952 >> v2:172.21.15.47:3300/0 conn(0x7f4c900987a0 0x7f4c90137930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.369+0000 7f4c8ed9d640 1 --2- 172.21.15.47:0/2445418952 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c9012f3c0 0x7f4c9012f7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c90099780 tx=0x7f4c8002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccbe5f5619d3f35d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.369+0000 7f4c8dd9b640 1 -- 172.21.15.47:0/2445418952 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80037070 con 0x7f4c9012f3c0 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.370+0000 7f4c8dd9b640 1 -- 172.21.15.47:0/2445418952 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8002adb0 con 0x7f4c9012f3c0 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.370+0000 7f4c8dd9b640 1 -- 172.21.15.47:0/2445418952 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80032660 con 0x7f4c9012f3c0 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.370+0000 7f4c95055640 1 -- 172.21.15.47:0/2445418952 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c9012f3c0 msgr2=0x7f4c9012f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.370+0000 7f4c95055640 1 --2- 172.21.15.47:0/2445418952 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c9012f3c0 0x7f4c9012f7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c90099780 tx=0x7f4c8002a060 comp rx=0 tx=0).stop 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.370+0000 7f4c95055640 1 -- 172.21.15.47:0/2445418952 shutdown_connections 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.370+0000 7f4c95055640 1 --2- 172.21.15.47:0/2445418952 >> v2:172.21.15.47:3300/0 conn(0x7f4c900987a0 0x7f4c90137930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.370+0000 7f4c95055640 1 --2- 172.21.15.47:0/2445418952 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c9012f3c0 0x7f4c9012f7a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.370+0000 7f4c95055640 1 -- 172.21.15.47:0/2445418952 >> 172.21.15.47:0/2445418952 conn(0x7f4c900a23b0 msgr2=0x7f4c9009e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.370+0000 7f4c95055640 1 -- 172.21.15.47:0/2445418952 shutdown_connections 2024-09-15T21:56:47.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.370+0000 7f4c95055640 1 -- 172.21.15.47:0/2445418952 wait complete. 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.371+0000 7f4c95055640 1 Processor -- start 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.371+0000 7f4c95055640 1 -- start start 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c95055640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c900987a0 0x7f4c901c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c95055640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4c9012f3c0 0x7f4c901c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c95055640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c9012fce0 0x7f4c9008cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c95055640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90098c10 con 0x7f4c9012fce0 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c95055640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9006ee60 con 0x7f4c900987a0 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c95055640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f4c90098d90 con 0x7f4c9012f3c0 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8ed9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c900987a0 0x7f4c901c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8e59c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4c9012f3c0 0x7f4c901c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8f59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c9012fce0 0x7f4c9008cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:47.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8ed9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c900987a0 0x7f4c901c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:59500/0 (socket says 172.21.15.47:59500) 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8ed9d640 1 -- 172.21.15.47:0/3900135154 learned_addr learned my addr 172.21.15.47:0/3900135154 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8ed9d640 1 -- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4c9012f3c0 msgr2=0x7f4c901c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8ed9d640 1 --2- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4c9012f3c0 0x7f4c901c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8ed9d640 1 -- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c9012fce0 msgr2=0x7f4c9008cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8ed9d640 1 --2- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c9012fce0 0x7f4c9008cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8ed9d640 1 -- 172.21.15.47:0/3900135154 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c90090500 con 0x7f4c900987a0 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8f59e640 1 --2- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c9012fce0 0x7f4c9008cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8e59c640 1 --2- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4c9012f3c0 0x7f4c901c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.372+0000 7f4c8ed9d640 1 --2- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c900987a0 0x7f4c901c4960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c80000c00 tx=0x7f4c800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.373+0000 7f4c777fe640 1 -- 172.21.15.47:0/3900135154 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80041070 con 0x7f4c900987a0 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.373+0000 7f4c777fe640 1 -- 172.21.15.47:0/3900135154 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8003c460 con 0x7f4c900987a0 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.373+0000 7f4c95055640 1 -- 172.21.15.47:0/3900135154 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9008f1a0 con 0x7f4c900987a0 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.373+0000 7f4c777fe640 1 -- 172.21.15.47:0/3900135154 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80037040 con 0x7f4c900987a0 2024-09-15T21:56:47.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.373+0000 7f4c95055640 1 -- 172.21.15.47:0/3900135154 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9008fb30 con 0x7f4c900987a0 2024-09-15T21:56:47.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.374+0000 7f4c95055640 1 -- 172.21.15.47:0/3900135154 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9012f7a0 con 0x7f4c900987a0 2024-09-15T21:56:47.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.374+0000 7f4c777fe640 1 -- 172.21.15.47:0/3900135154 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f4c80044070 con 0x7f4c900987a0 2024-09-15T21:56:47.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.374+0000 7f4c777fe640 1 -- 172.21.15.47:0/3900135154 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 3023+0+0 (secure 0 0 0) 0x7f4c8007df30 con 0x7f4c900987a0 2024-09-15T21:56:47.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.378+0000 7f4c777fe640 1 -- 172.21.15.47:0/3900135154 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f4c8007cff0 con 0x7f4c900987a0 2024-09-15T21:56:47.482 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-09-15T21:56:47.481+0000 7fcc05ddd640 -1 osd.3 22 log_to_monitors true 2024-09-15T21:56:47.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.531+0000 7f4c95055640 1 -- 172.21.15.47:0/3900135154 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c90068e40 con 0x7f4c900987a0 2024-09-15T21:56:47.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.532+0000 7f4c777fe640 1 -- 172.21.15.47:0/3900135154 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+9019 (secure 0 0 0) 0x7f4c8007cb30 con 0x7f4c900987a0 2024-09-15T21:56:47.533 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:47.533 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":25,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:56:46.704404+0000","last_up_change":"2024-09-15T21:56:46.704404+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"error_message":"pool 2 has some PGs where all OSDs are with primary_affinity 0 (at least pg 2.1), read balance score may not be reliable","score_acting":0,"score_stable":0}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":0,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":2079261393}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":2079261393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":2079261393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":2079261393}]},"public_addr":"172.21.15.47:6801/2079261393","cluster_addr":"172.21.15.47:6802/2079261393","heartbeat_back_addr":"172.21.15.47:6804/2079261393","heartbeat_front_addr":"172.21.15.47:6803/2079261393","state":["exists","up"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":25,"up_thru":20,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":4287109391}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":4287109391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":4287109391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":4287109391}]},"public_addr":"172.21.15.47:6805/4287109391","cluster_addr":"172.21.15.47:6806/4287109391","heartbeat_back_addr":"172.21.15.47:6808/4287109391","heartbeat_front_addr":"172.21.15.47:6807/4287109391","state":["exists","up"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":32772}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":32772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":32772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":32772}]},"public_addr":"172.21.15.47:6813/32772","cluster_addr":"172.21.15.47:6814/32772","heartbeat_back_addr":"172.21.15.47:6816/32772","heartbeat_front_addr":"172.21.15.47:6815/32772","state":["exists"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":32770}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":32770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":32770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":32770}]},"public_addr":"172.21.15.47:6809/32770","cluster_addr":"172.21.15.47:6810/32770","heartbeat_back_addr":"172.21.15.47:6812/32770","heartbeat_front_addr":"172.21.15.47:6811/32770","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-15T21:56:40.690629+0000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/3200219561":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/2444109955":"2024-09-16T21:56:33.661886+0000","172.21.15.47:6800/32267":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1739125583":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1981014788":"2024-09-16T21:56:33.661886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:56:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.533+0000 7f4c95055640 1 -- 172.21.15.47:0/3900135154 >> v1:172.21.15.47:6800/1790418198 conn(0x7f4c7c048000 legacy=0x7f4c7c04a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.533+0000 7f4c95055640 1 -- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c900987a0 msgr2=0x7f4c901c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.533+0000 7f4c95055640 1 --2- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c900987a0 0x7f4c901c4960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c80000c00 tx=0x7f4c800046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.533+0000 7f4c95055640 1 -- 172.21.15.47:0/3900135154 shutdown_connections 2024-09-15T21:56:47.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.533+0000 7f4c95055640 1 --2- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c9012fce0 0x7f4c9008cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:47.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.533+0000 7f4c95055640 1 --2- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4c9012f3c0 0x7f4c901c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:47.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.533+0000 7f4c95055640 1 --2- 172.21.15.47:0/3900135154 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4c900987a0 0x7f4c901c4960 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:47.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.533+0000 7f4c95055640 1 -- 172.21.15.47:0/3900135154 >> 172.21.15.47:0/3900135154 conn(0x7f4c900a23b0 msgr2=0x7f4c90099b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:47.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.533+0000 7f4c95055640 1 -- 172.21.15.47:0/3900135154 shutdown_connections 2024-09-15T21:56:47.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:47.533+0000 7f4c95055640 1 -- 172.21.15.47:0/3900135154 wait complete. 2024-09-15T21:56:47.716 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-09-15T21:56:47.715+0000 7f2212efd640 -1 osd.2 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:56:47.717 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-09-15T21:56:47.716+0000 7fcbf9b7c640 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:56:50.546 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:56:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.658+0000 7fd953ba3640 1 Processor -- start 2024-09-15T21:56:50.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.659+0000 7fd953ba3640 1 -- start start 2024-09-15T21:56:50.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.659+0000 7fd953ba3640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12f3c0 0x7fd94c12f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.659+0000 7fd953ba3640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12fce0 0x7fd94c098090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.659+0000 7fd953ba3640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94c09f230 con 0x7fd94c0985d0 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.659+0000 7fd953ba3640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c06f260 con 0x7fd94c12f3c0 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.659+0000 7fd953ba3640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd94c099600 con 0x7fd94c12fce0 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.659+0000 7fd952140640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd94c0985d0 0x7fd94c136920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34258/0 (socket says 172.21.15.47:34258) 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.659+0000 7fd952140640 1 -- 172.21.15.47:0/2598074929 learned_addr learned my addr 172.21.15.47:0/2598074929 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.659+0000 7fd95193f640 1 --2- 172.21.15.47:0/2598074929 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12f3c0 0x7fd94c12f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.659+0000 7fd95113e640 1 --2- 172.21.15.47:0/2598074929 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12fce0 0x7fd94c098090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.660+0000 7fd95093d640 1 -- 172.21.15.47:0/2598074929 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2872488111 0 0) 0x7fd94c09f230 con 0x7fd94c0985d0 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.660+0000 7fd95113e640 1 -- 172.21.15.47:0/2598074929 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12f3c0 msgr2=0x7fd94c12f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:50.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.660+0000 7fd95113e640 1 --2- 172.21.15.47:0/2598074929 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12f3c0 0x7fd94c12f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.660+0000 7fd95113e640 1 -- 172.21.15.47:0/2598074929 >> v1:172.21.15.47:6789/0 conn(0x7fd94c0985d0 legacy=0x7fd94c136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.660+0000 7fd95113e640 1 -- 172.21.15.47:0/2598074929 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c130120 con 0x7fd94c12fce0 2024-09-15T21:56:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.660+0000 7fd95193f640 1 --2- 172.21.15.47:0/2598074929 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12f3c0 0x7fd94c12f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:50.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.660+0000 7fd95113e640 1 --2- 172.21.15.47:0/2598074929 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12fce0 0x7fd94c098090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0097d0 tx=0x7fd93c02a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8d510b4752adbd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd95093d640 1 -- 172.21.15.47:0/2598074929 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c037070 con 0x7fd94c12fce0 2024-09-15T21:56:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd95093d640 1 -- 172.21.15.47:0/2598074929 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd93c02ad90 con 0x7fd94c12fce0 2024-09-15T21:56:50.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd95093d640 1 -- 172.21.15.47:0/2598074929 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c032660 con 0x7fd94c12fce0 2024-09-15T21:56:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd953ba3640 1 -- 172.21.15.47:0/2598074929 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12fce0 msgr2=0x7fd94c098090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd953ba3640 1 --2- 172.21.15.47:0/2598074929 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12fce0 0x7fd94c098090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd93c0097d0 tx=0x7fd93c02a040 comp rx=0 tx=0).stop 2024-09-15T21:56:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd953ba3640 1 -- 172.21.15.47:0/2598074929 shutdown_connections 2024-09-15T21:56:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd953ba3640 1 --2- 172.21.15.47:0/2598074929 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12fce0 0x7fd94c098090 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd953ba3640 1 --2- 172.21.15.47:0/2598074929 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12f3c0 0x7fd94c12f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd953ba3640 1 -- 172.21.15.47:0/2598074929 >> 172.21.15.47:0/2598074929 conn(0x7fd94c0a23b0 msgr2=0x7fd94c09e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd953ba3640 1 -- 172.21.15.47:0/2598074929 shutdown_connections 2024-09-15T21:56:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.661+0000 7fd953ba3640 1 -- 172.21.15.47:0/2598074929 wait complete. 2024-09-15T21:56:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.662+0000 7fd953ba3640 1 Processor -- start 2024-09-15T21:56:50.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.662+0000 7fd953ba3640 1 -- start start 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd953ba3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd94c0985d0 0x7fd94c1c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd953ba3640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12f3c0 0x7fd94c1c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd953ba3640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12fce0 0x7fd94c08cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd953ba3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c098a50 con 0x7fd94c0985d0 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd953ba3640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c06ee60 con 0x7fd94c12fce0 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd953ba3640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd94c098bd0 con 0x7fd94c12f3c0 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95113e640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12f3c0 0x7fd94c1c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95193f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd94c0985d0 0x7fd94c1c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd952140640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12fce0 0x7fd94c08cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95113e640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12f3c0 0x7fd94c1c4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:41878/0 (socket says 172.21.15.47:41878) 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95113e640 1 -- 172.21.15.47:0/1174817685 learned_addr learned my addr 172.21.15.47:0/1174817685 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:50.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95193f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd94c0985d0 0x7fd94c1c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45688/0 (socket says 172.21.15.47:45688) 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd952140640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12fce0 0x7fd94c08cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44520/0 (socket says 172.21.15.47:44520) 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95113e640 1 -- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12fce0 msgr2=0x7fd94c08cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95113e640 1 --2- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12fce0 0x7fd94c08cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95113e640 1 -- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd94c0985d0 msgr2=0x7fd94c1c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95113e640 1 --2- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd94c0985d0 0x7fd94c1c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95113e640 1 -- 172.21.15.47:0/1174817685 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c08f870 con 0x7fd94c12f3c0 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95193f640 1 --2- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd94c0985d0 0x7fd94c1c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd95113e640 1 --2- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12f3c0 0x7fd94c1c4cc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd93c035040 tx=0x7fd93c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd9427fc640 1 -- 172.21.15.47:0/1174817685 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c037050 con 0x7fd94c12f3c0 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd9427fc640 1 -- 172.21.15.47:0/1174817685 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd93c038040 con 0x7fd94c12f3c0 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd953ba3640 1 -- 172.21.15.47:0/1174817685 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c08f030 con 0x7fd94c12f3c0 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.663+0000 7fd9427fc640 1 -- 172.21.15.47:0/1174817685 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c004270 con 0x7fd94c12f3c0 2024-09-15T21:56:50.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.664+0000 7fd953ba3640 1 -- 172.21.15.47:0/1174817685 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c1c53e0 con 0x7fd94c12f3c0 2024-09-15T21:56:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.664+0000 7fd9427fc640 1 -- 172.21.15.47:0/1174817685 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fd93c003610 con 0x7fd94c12f3c0 2024-09-15T21:56:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.665+0000 7fd953ba3640 1 -- 172.21.15.47:0/1174817685 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c12f7a0 con 0x7fd94c12f3c0 2024-09-15T21:56:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.665+0000 7fd9427fc640 1 -- 172.21.15.47:0/1174817685 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(28..28 src has 1..28) ==== 3023+0+0 (secure 0 0 0) 0x7fd93c07df50 con 0x7fd94c12f3c0 2024-09-15T21:56:50.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.669+0000 7fd9427fc640 1 -- 172.21.15.47:0/1174817685 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fd93c07d010 con 0x7fd94c12f3c0 2024-09-15T21:56:50.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.819+0000 7fd953ba3640 1 -- 172.21.15.47:0/1174817685 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd94c068e40 con 0x7fd94c12f3c0 2024-09-15T21:56:50.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.820+0000 7fd9427fc640 1 -- 172.21.15.47:0/1174817685 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+9173 (secure 0 0 0) 0x7fd93c004410 con 0x7fd94c12f3c0 2024-09-15T21:56:50.821 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:50.821 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":28,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:56:49.722337+0000","last_up_change":"2024-09-15T21:56:48.715593+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":2079261393}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":2079261393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":2079261393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":2079261393}]},"public_addr":"172.21.15.47:6801/2079261393","cluster_addr":"172.21.15.47:6802/2079261393","heartbeat_back_addr":"172.21.15.47:6804/2079261393","heartbeat_front_addr":"172.21.15.47:6803/2079261393","state":["exists","up"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":25,"up_thru":25,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":4287109391}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":4287109391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":4287109391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":4287109391}]},"public_addr":"172.21.15.47:6805/4287109391","cluster_addr":"172.21.15.47:6806/4287109391","heartbeat_back_addr":"172.21.15.47:6808/4287109391","heartbeat_front_addr":"172.21.15.47:6807/4287109391","state":["exists","up"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":27,"up_thru":27,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":2903715449}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":2903715449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":2903715449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":2903715449}]},"public_addr":"172.21.15.47:6809/2903715449","cluster_addr":"172.21.15.47:6810/2903715449","heartbeat_back_addr":"172.21.15.47:6812/2903715449","heartbeat_front_addr":"172.21.15.47:6811/2903715449","state":["exists","up"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":27,"up_thru":27,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":2978958959}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":2978958959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":2978958959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":2978958959}]},"public_addr":"172.21.15.47:6813/2978958959","cluster_addr":"172.21.15.47:6814/2978958959","heartbeat_back_addr":"172.21.15.47:6816/2978958959","heartbeat_front_addr":"172.21.15.47:6815/2978958959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-15T21:56:40.690629+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/3200219561":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/2444109955":"2024-09-16T21:56:33.661886+0000","172.21.15.47:6800/32267":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1739125583":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1981014788":"2024-09-16T21:56:33.661886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:56:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.821+0000 7fd953ba3640 1 -- 172.21.15.47:0/1174817685 >> v1:172.21.15.47:6800/1790418198 conn(0x7fd930047fd0 legacy=0x7fd93004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.821+0000 7fd953ba3640 1 -- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12f3c0 msgr2=0x7fd94c1c4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:50.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.821+0000 7fd953ba3640 1 --2- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12f3c0 0x7fd94c1c4cc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd93c035040 tx=0x7fd93c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.822+0000 7fd953ba3640 1 -- 172.21.15.47:0/1174817685 shutdown_connections 2024-09-15T21:56:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.822+0000 7fd953ba3640 1 --2- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd94c12fce0 0x7fd94c08cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.822+0000 7fd953ba3640 1 --2- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd94c12f3c0 0x7fd94c1c4cc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.822+0000 7fd953ba3640 1 --2- 172.21.15.47:0/1174817685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd94c0985d0 0x7fd94c1c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.822+0000 7fd953ba3640 1 -- 172.21.15.47:0/1174817685 >> 172.21.15.47:0/1174817685 conn(0x7fd94c0a23b0 msgr2=0x7fd94c09a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.822+0000 7fd953ba3640 1 -- 172.21.15.47:0/1174817685 shutdown_connections 2024-09-15T21:56:50.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.822+0000 7fd953ba3640 1 -- 172.21.15.47:0/1174817685 wait complete. 2024-09-15T21:56:50.834 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:56:50.834 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T21:56:50.846 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T21:56:50.846 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:56:50.846 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph versions' 2024-09-15T21:56:50.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b2d765640 1 Processor -- start 2024-09-15T21:56:50.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b2d765640 1 -- start start 2024-09-15T21:56:50.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b2d765640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806d130 0x7f4b2806d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b2d765640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806da50 0x7f4b2812f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b2d765640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b2806e250 con 0x7f4b2812f770 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b2d765640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2806ec40 con 0x7f4b2806da50 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b2d765640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f4b28097890 con 0x7f4b2806d130 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b277fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4b2812f770 0x7f4b2812fbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34260/0 (socket says 172.21.15.47:34260) 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b267fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806da50 0x7f4b2812f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b277fe640 1 -- 172.21.15.47:0/1597449429 learned_addr learned my addr 172.21.15.47:0/1597449429 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.943+0000 7f4b26ffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806d130 0x7f4b2806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.944+0000 7f4b25ffb640 1 -- 172.21.15.47:0/1597449429 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718969369 0 0) 0x7f4b2806e250 con 0x7f4b2812f770 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.944+0000 7f4b26ffd640 1 -- 172.21.15.47:0/1597449429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806da50 msgr2=0x7f4b2812f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.944+0000 7f4b26ffd640 1 --2- 172.21.15.47:0/1597449429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806da50 0x7f4b2812f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.944+0000 7f4b26ffd640 1 -- 172.21.15.47:0/1597449429 >> v1:172.21.15.47:6789/0 conn(0x7f4b2812f770 legacy=0x7f4b2812fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:50.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.944+0000 7f4b26ffd640 1 -- 172.21.15.47:0/1597449429 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2806ded0 con 0x7f4b2806d130 2024-09-15T21:56:50.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.944+0000 7f4b267fc640 1 --2- 172.21.15.47:0/1597449429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806da50 0x7f4b2812f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.944+0000 7f4b26ffd640 1 --2- 172.21.15.47:0/1597449429 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806d130 0x7f4b2806d510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4b180097d0 tx=0x7f4b1802a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c674573066426193 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b25ffb640 1 -- 172.21.15.47:0/1597449429 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18037070 con 0x7f4b2806d130 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b25ffb640 1 -- 172.21.15.47:0/1597449429 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1802ae20 con 0x7f4b2806d130 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b25ffb640 1 -- 172.21.15.47:0/1597449429 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18032660 con 0x7f4b2806d130 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b2d765640 1 -- 172.21.15.47:0/1597449429 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806d130 msgr2=0x7f4b2806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b2d765640 1 --2- 172.21.15.47:0/1597449429 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806d130 0x7f4b2806d510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4b180097d0 tx=0x7f4b1802a0d0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b2d765640 1 -- 172.21.15.47:0/1597449429 shutdown_connections 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b2d765640 1 --2- 172.21.15.47:0/1597449429 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806da50 0x7f4b2812f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b2d765640 1 --2- 172.21.15.47:0/1597449429 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806d130 0x7f4b2806d510 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b2d765640 1 -- 172.21.15.47:0/1597449429 >> 172.21.15.47:0/1597449429 conn(0x7f4b2809fde0 msgr2=0x7f4b280a01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:50.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b2d765640 1 -- 172.21.15.47:0/1597449429 shutdown_connections 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.945+0000 7f4b2d765640 1 -- 172.21.15.47:0/1597449429 wait complete. 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.946+0000 7f4b2d765640 1 Processor -- start 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.946+0000 7f4b2d765640 1 -- start start 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b2d765640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806d130 0x7f4b28124cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b2d765640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806da50 0x7f4b28125200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b2d765640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b2812f770 0x7f4b28125740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b2d765640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28137960 con 0x7f4b2812f770 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b2d765640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f4b281377e0 con 0x7f4b2806d130 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b2d765640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f4b28137ae0 con 0x7f4b2806da50 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b26ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806d130 0x7f4b28124cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b2812f770 0x7f4b28125740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b267fc640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806da50 0x7f4b28125200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b26ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806d130 0x7f4b28124cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44546/0 (socket says 172.21.15.47:44546) 2024-09-15T21:56:50.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b26ffd640 1 -- 172.21.15.47:0/2053391499 learned_addr learned my addr 172.21.15.47:0/2053391499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b277fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b2812f770 0x7f4b28125740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45694/0 (socket says 172.21.15.47:45694) 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b26ffd640 1 -- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806da50 msgr2=0x7f4b28125200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b26ffd640 1 --2- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806da50 0x7f4b28125200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b26ffd640 1 -- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b2812f770 msgr2=0x7f4b28125740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b26ffd640 1 --2- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b2812f770 0x7f4b28125740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b26ffd640 1 -- 172.21.15.47:0/2053391499 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b28125ea0 con 0x7f4b2806d130 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b267fc640 1 --2- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806da50 0x7f4b28125200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b277fe640 1 --2- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b2812f770 0x7f4b28125740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.947+0000 7f4b26ffd640 1 --2- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806d130 0x7f4b28124cc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b180098f0 tx=0x7f4b180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.948+0000 7f4b0f7fe640 1 -- 172.21.15.47:0/2053391499 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18041030 con 0x7f4b2806d130 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.948+0000 7f4b0f7fe640 1 -- 172.21.15.47:0/2053391499 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1802ac20 con 0x7f4b2806d130 2024-09-15T21:56:50.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.948+0000 7f4b0f7fe640 1 -- 172.21.15.47:0/2053391499 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18037040 con 0x7f4b2806d130 2024-09-15T21:56:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.948+0000 7f4b2d765640 1 -- 172.21.15.47:0/2053391499 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b28099f90 con 0x7f4b2806d130 2024-09-15T21:56:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.948+0000 7f4b2d765640 1 -- 172.21.15.47:0/2053391499 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b28136f80 con 0x7f4b2806d130 2024-09-15T21:56:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.949+0000 7f4b2d765640 1 -- 172.21.15.47:0/2053391499 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2806d510 con 0x7f4b2806d130 2024-09-15T21:56:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.949+0000 7f4b0f7fe640 1 -- 172.21.15.47:0/2053391499 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f4b18044070 con 0x7f4b2806d130 2024-09-15T21:56:50.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.949+0000 7f4b0f7fe640 1 -- 172.21.15.47:0/2053391499 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 3023+0+0 (secure 0 0 0) 0x7f4b1807ded0 con 0x7f4b2806d130 2024-09-15T21:56:50.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:50.953+0000 7f4b0f7fe640 1 -- 172.21.15.47:0/2053391499 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f4b1807cf90 con 0x7f4b2806d130 2024-09-15T21:56:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.152+0000 7f4b2d765640 1 -- 172.21.15.47:0/2053391499 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f4b28132a80 con 0x7f4b2806d130 2024-09-15T21:56:51.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.153+0000 7f4b0f7fe640 1 -- 172.21.15.47:0/2053391499 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+507 (secure 0 0 0) 0x7f4b1802bd50 con 0x7f4b2806d130 2024-09-15T21:56:51.154 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": { 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 3 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": { 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 2 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": { 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 4 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: "overall": { 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)": 9 2024-09-15T21:56:51.155 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.156 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T21:56:51.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.154+0000 7f4b2d765640 1 -- 172.21.15.47:0/2053391499 >> v1:172.21.15.47:6800/1790418198 conn(0x7f4b14047fd0 legacy=0x7f4b1404a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.154+0000 7f4b2d765640 1 -- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806d130 msgr2=0x7f4b28124cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.154+0000 7f4b2d765640 1 --2- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806d130 0x7f4b28124cc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4b180098f0 tx=0x7f4b180046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.154+0000 7f4b2d765640 1 -- 172.21.15.47:0/2053391499 shutdown_connections 2024-09-15T21:56:51.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.154+0000 7f4b2d765640 1 --2- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b2812f770 0x7f4b28125740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.154+0000 7f4b2d765640 1 --2- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4b2806da50 0x7f4b28125200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.154+0000 7f4b2d765640 1 --2- 172.21.15.47:0/2053391499 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4b2806d130 0x7f4b28124cc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.154+0000 7f4b2d765640 1 -- 172.21.15.47:0/2053391499 >> 172.21.15.47:0/2053391499 conn(0x7f4b2809fde0 msgr2=0x7f4b28099110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:51.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.155+0000 7f4b2d765640 1 -- 172.21.15.47:0/2053391499 shutdown_connections 2024-09-15T21:56:51.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.155+0000 7f4b2d765640 1 -- 172.21.15.47:0/2053391499 wait complete. 2024-09-15T21:56:51.167 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd dump -f json-pretty' 2024-09-15T21:56:51.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.264+0000 7fa686b11640 1 Processor -- start 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.264+0000 7fa686b11640 1 -- start start 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.264+0000 7fa686b11640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680097930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.264+0000 7fa686b11640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa68006fcb0 0x7fa68012bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa686b11640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa680099e00 con 0x7fa68012c4f0 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa686b11640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa6800708f0 con 0x7fa68006fcb0 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa686b11640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fa68006e0c0 con 0x7fa680097550 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa6848ad640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680097930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa67ffff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa68006fcb0 0x7fa68012bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa6848ad640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680097930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44554/0 (socket says 172.21.15.47:44554) 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa6850ae640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa68012c4f0 0x7fa6801332e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34272/0 (socket says 172.21.15.47:34272) 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa6848ad640 1 -- 172.21.15.47:0/2513453089 learned_addr learned my addr 172.21.15.47:0/2513453089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:51.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa67ffff640 1 -- 172.21.15.47:0/2513453089 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 msgr2=0x7fa680097930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa67ffff640 1 --2- 172.21.15.47:0/2513453089 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680097930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa67ffff640 1 -- 172.21.15.47:0/2513453089 >> v1:172.21.15.47:6789/0 conn(0x7fa68012c4f0 legacy=0x7fa6801332e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa67ffff640 1 -- 172.21.15.47:0/2513453089 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa680070490 con 0x7fa68006fcb0 2024-09-15T21:56:51.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.265+0000 7fa6848ad640 1 --2- 172.21.15.47:0/2513453089 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680097930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa67ffff640 1 --2- 172.21.15.47:0/2513453089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68006fcb0 0x7fa68012bfb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6740097a0 tx=0x7fa67402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2b714169871742 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa67f7fe640 1 -- 172.21.15.47:0/2513453089 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674037070 con 0x7fa68006fcb0 2024-09-15T21:56:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa67f7fe640 1 -- 172.21.15.47:0/2513453089 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67402adb0 con 0x7fa68006fcb0 2024-09-15T21:56:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa67f7fe640 1 -- 172.21.15.47:0/2513453089 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674032660 con 0x7fa68006fcb0 2024-09-15T21:56:51.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa686b11640 1 -- 172.21.15.47:0/2513453089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68006fcb0 msgr2=0x7fa68012bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa686b11640 1 --2- 172.21.15.47:0/2513453089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68006fcb0 0x7fa68012bfb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6740097a0 tx=0x7fa67402a060 comp rx=0 tx=0).stop 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa686b11640 1 -- 172.21.15.47:0/2513453089 shutdown_connections 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa686b11640 1 --2- 172.21.15.47:0/2513453089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68006fcb0 0x7fa68012bfb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa686b11640 1 --2- 172.21.15.47:0/2513453089 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680097930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa686b11640 1 -- 172.21.15.47:0/2513453089 >> 172.21.15.47:0/2513453089 conn(0x7fa68009aba0 msgr2=0x7fa680098470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa686b11640 1 -- 172.21.15.47:0/2513453089 shutdown_connections 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.266+0000 7fa686b11640 1 -- 172.21.15.47:0/2513453089 wait complete. 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.267+0000 7fa686b11640 1 Processor -- start 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.267+0000 7fa686b11640 1 -- start start 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa686b11640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa68006fcb0 0x7fa680135a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa686b11640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680135f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa686b11640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68012c4f0 0x7fa680136490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa686b11640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa68012cb00 con 0x7fa68012c4f0 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa686b11640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fa68012c980 con 0x7fa680097550 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa686b11640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fa68012cc80 con 0x7fa68006fcb0 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa6848ad640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa68006fcb0 0x7fa680135a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa67ffff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680135f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa6848ad640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa68006fcb0 0x7fa680135a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:41896/0 (socket says 172.21.15.47:41896) 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa67ffff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680135f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44560/0 (socket says 172.21.15.47:44560) 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa6848ad640 1 -- 172.21.15.47:0/3440944395 learned_addr learned my addr 172.21.15.47:0/3440944395 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa6850ae640 1 --2- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68012c4f0 0x7fa680136490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa6850ae640 1 -- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa68006fcb0 msgr2=0x7fa680135a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa6850ae640 1 --2- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa68006fcb0 0x7fa680135a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa6850ae640 1 -- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 msgr2=0x7fa680135f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa6850ae640 1 --2- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680135f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa6850ae640 1 -- 172.21.15.47:0/3440944395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68006f7b0 con 0x7fa68012c4f0 2024-09-15T21:56:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa67ffff640 1 --2- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680135f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.268+0000 7fa6850ae640 1 --2- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68012c4f0 0x7fa680136490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa67800c6c0 tx=0x7fa67800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.269+0000 7fa67d7fa640 1 -- 172.21.15.47:0/3440944395 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678018070 con 0x7fa68012c4f0 2024-09-15T21:56:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.269+0000 7fa67d7fa640 1 -- 172.21.15.47:0/3440944395 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa678004d60 con 0x7fa68012c4f0 2024-09-15T21:56:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.269+0000 7fa67d7fa640 1 -- 172.21.15.47:0/3440944395 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67800a840 con 0x7fa68012c4f0 2024-09-15T21:56:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.269+0000 7fa686b11640 1 -- 172.21.15.47:0/3440944395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa680128fc0 con 0x7fa68012c4f0 2024-09-15T21:56:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.269+0000 7fa686b11640 1 -- 172.21.15.47:0/3440944395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6801c49b0 con 0x7fa68012c4f0 2024-09-15T21:56:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.270+0000 7fa686b11640 1 -- 172.21.15.47:0/3440944395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6800950c0 con 0x7fa68012c4f0 2024-09-15T21:56:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.270+0000 7fa67d7fa640 1 -- 172.21.15.47:0/3440944395 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fa67801b070 con 0x7fa68012c4f0 2024-09-15T21:56:51.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.270+0000 7fa67d7fa640 1 -- 172.21.15.47:0/3440944395 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3023+0+0 (secure 0 0 0) 0x7fa67805cf00 con 0x7fa68012c4f0 2024-09-15T21:56:51.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.274+0000 7fa67d7fa640 1 -- 172.21.15.47:0/3440944395 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fa67805bfc0 con 0x7fa68012c4f0 2024-09-15T21:56:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.420+0000 7fa686b11640 1 -- 172.21.15.47:0/3440944395 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fa68012bfb0 con 0x7fa68012c4f0 2024-09-15T21:56:51.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.421+0000 7fa67d7fa640 1 -- 172.21.15.47:0/3440944395 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v28) ==== 81+0+16837 (secure 0 0 0) 0x7fa67803c100 con 0x7fa68012c4f0 2024-09-15T21:56:51.422 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 28, 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "fsid": "519e4c2c-d517-4539-a86b-3d427c67bae0", 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "created": "2024-09-15T21:55:00.295766+0000", 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "modified": "2024-09-15T21:56:49.722337+0000", 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "last_up_change": "2024-09-15T21:56:48.715593+0000", 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "last_in_change": "2024-09-15T21:55:02.238284+0000", 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_num": 5799936, 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_set": [ 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "pglog_hardlimit", 2024-09-15T21:56:51.423 INFO:teuthology.orchestra.run.smithi047.stdout: "purged_snapdirs", 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_deletes", 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "sortbitwise" 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_version": 6, 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_snapshot": "", 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_max": 2, 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd": 4, 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "require_min_compat_client": "quincy", 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "min_compat_client": "jewel", 2024-09-15T21:56:51.424 INFO:teuthology.orchestra.run.smithi047.stdout: "require_osd_release": "quincy", 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_crimson": false, 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "pools": [ 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 1, 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": ".mgr", 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2024-09-15T21:55:10.740066+0000", 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool", 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:56:51.425 INFO:teuthology.orchestra.run.smithi047.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "is_stretch_pool": false, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 1, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 1, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 1, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 1, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2024-09-15T21:56:51.426 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "12", 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 0, 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 0, 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2024-09-15T21:56:51.427 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2024-09-15T21:56:51.428 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2024-09-15T21:56:51.429 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "options": { 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_max": 32, 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_min": 1 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": {} 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "read_balance": { 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "score_type": "Fair distribution", 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "score_acting": 4, 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "score_stable": 4, 2024-09-15T21:56:51.430 INFO:teuthology.orchestra.run.smithi047.stdout: "optimal_score": 0.5, 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "raw_score_acting": 2, 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "raw_score_stable": 2, 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity_weighted": 1, 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "average_primary_affinity": 1, 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "pool": 2, 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": "rbd", 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "create_time": "2024-09-15T21:55:16.666135+0000", 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 8193, 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T21:56:51.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": 1, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "size": 2, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": 1, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_rule": 0, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "is_stretch_pool": false, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 8, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num": 8, 2024-09-15T21:56:51.432 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_placement_num_target": 8, 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_target": 8, 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num_pending": 8, 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "last_pg_merge_meta": { 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "source_pgid": "0.0", 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "ready_epoch": 0, 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_started": 0, 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "last_epoch_clean": 0, 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "source_version": "0'0", 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "target_version": "0'0" 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "last_change": "16", 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend": "0", 2024-09-15T21:56:51.433 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "auid": 0, 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_seq": 3, 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "snap_epoch": 16, 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_snaps": [], 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps": "[]", 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_bytes": 0, 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "read_tier": -1, 2024-09-15T21:56:51.434 INFO:teuthology.orchestra.run.smithi047.stdout: "write_tier": -1, 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_mode": "none", 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_bytes": 0, 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "target_max_objects": 0, 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_flush_age": 0, 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "cache_min_evict_age": 0, 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profile": "", 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_params": { 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "none" 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.435 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_period": 0, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_count": 0, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "use_gmt_hitset": true, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "grade_table": [], 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "stripe_width": 0, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "expected_num_objects": 0, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "fast_read": false, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { 2024-09-15T21:56:51.436 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd": {} 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: "read_balance": { 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: "score_type": "Fair distribution", 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: "score_acting": 2.5, 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: "score_stable": 2.5, 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: "optimal_score": 1, 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: "raw_score_acting": 2.5, 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: "raw_score_stable": 2.5, 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity_weighted": 1, 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: "average_primary_affinity": 1, 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.437 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "osds": [ 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "3f514164-0bf2-4d96-99c2-dc948a0d3cb0", 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 9, 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 19, 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 25, 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 25, 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 20, 2024-09-15T21:56:51.438 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6801", 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2079261393 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6802", 2024-09-15T21:56:51.439 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2079261393 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6804", 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2079261393 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2024-09-15T21:56:51.440 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6803", 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2079261393 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6801/2079261393", 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6802/2079261393", 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6804/2079261393", 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6803/2079261393", 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2024-09-15T21:56:51.441 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "98cbb40c-7873-4bb4-8e27-42dd62549685", 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 9, 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 20, 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 25, 2024-09-15T21:56:51.442 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 25, 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 21, 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6805", 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4287109391 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.443 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6806", 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4287109391 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6808", 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4287109391 2024-09-15T21:56:51.444 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6807", 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4287109391 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6805/4287109391", 2024-09-15T21:56:51.445 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6806/4287109391", 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6808/4287109391", 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6807/4287109391", 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "cd82ae94-f43c-476f-bb97-a0699b859536", 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2024-09-15T21:56:51.446 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 9, 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 21, 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 27, 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 27, 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 22, 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6809", 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2903715449 2024-09-15T21:56:51.447 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6810", 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2903715449 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.448 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6812", 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2903715449 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.449 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6811", 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2903715449 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6809/2903715449", 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6810/2903715449", 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6812/2903715449", 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6811/2903715449", 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.450 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "uuid": "708e9506-4b05-4c61-95fa-a920cdc9d8fa", 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "up": 1, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "in": 1, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "weight": 1, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_affinity": 1, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_begin": 9, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "last_clean_end": 22, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "up_from": 27, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "up_thru": 27, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "down_at": 23, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "lost_at": 0, 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addrs": { 2024-09-15T21:56:51.451 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6813", 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2978958959 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6814", 2024-09-15T21:56:51.452 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2978958959 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addrs": { 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6816", 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2978958959 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addrs": { 2024-09-15T21:56:51.453 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6815", 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2978958959 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: "public_addr": "172.21.15.47:6813/2978958959", 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addr": "172.21.15.47:6814/2978958959", 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_back_addr": "172.21.15.47:6816/2978958959", 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: "heartbeat_front_addr": "172.21.15.47:6815/2978958959", 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: "state": [ 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: "exists", 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: "up" 2024-09-15T21:56:51.454 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_xinfo": [ 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 0, 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540701547738038271, 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2024-09-15T21:55:07.272012+0000", 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 20 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.455 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 1, 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540701547738038271, 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2024-09-15T21:55:07.321013+0000", 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 21 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 2, 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "0.000000", 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2024-09-15T21:56:51.456 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540701547738038271, 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2024-09-15T21:55:07.397110+0000", 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 22 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": 3, 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "down_stamp": "2024-09-15T21:56:40.690629+0000", 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_probability": 0, 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "laggy_interval": 0, 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "features": 4540701547738038271, 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "old_weight": 0, 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "last_purged_snaps_scrub": "2024-09-15T21:55:07.350708+0000", 2024-09-15T21:56:51.457 INFO:teuthology.orchestra.run.smithi047.stdout: "dead_epoch": 0 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap": [], 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_items": [], 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_upmap_primaries": [], 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_temp": [], 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "primary_temp": [], 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklist": { 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:0/3200219561": "2024-09-16T21:56:33.661886+0000", 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:0/2444109955": "2024-09-16T21:56:33.661886+0000", 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:6800/32267": "2024-09-16T21:56:33.661886+0000", 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:0/1739125583": "2024-09-16T21:56:33.661886+0000", 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: "172.21.15.47:0/1981014788": "2024-09-16T21:56:33.661886+0000" 2024-09-15T21:56:51.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "range_blocklist": {}, 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "erasure_code_profiles": { 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "default": { 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-failure-domain": "osd", 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "k": "2", 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "m": "1", 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "plugin": "jerasure", 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "technique": "reed_sol_van" 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "removed_snaps_queue": [], 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "new_removed_snaps": [], 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "new_purged_snaps": [], 2024-09-15T21:56:51.459 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_node_flags": {}, 2024-09-15T21:56:51.460 INFO:teuthology.orchestra.run.smithi047.stdout: "device_class_flags": {}, 2024-09-15T21:56:51.460 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode": { 2024-09-15T21:56:51.460 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode_enabled": false, 2024-09-15T21:56:51.460 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_bucket_count": 0, 2024-09-15T21:56:51.460 INFO:teuthology.orchestra.run.smithi047.stdout: "degraded_stretch_mode": 0, 2024-09-15T21:56:51.460 INFO:teuthology.orchestra.run.smithi047.stdout: "recovering_stretch_mode": 0, 2024-09-15T21:56:51.460 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode_bucket": 0 2024-09-15T21:56:51.460 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-15T21:56:51.460 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T21:56:51.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.423+0000 7fa686b11640 1 -- 172.21.15.47:0/3440944395 >> v1:172.21.15.47:6800/1790418198 conn(0x7fa66c048000 legacy=0x7fa66c04a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.423+0000 7fa686b11640 1 -- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68012c4f0 msgr2=0x7fa680136490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.423+0000 7fa686b11640 1 --2- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68012c4f0 0x7fa680136490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa67800c6c0 tx=0x7fa67800cb80 comp rx=0 tx=0).stop 2024-09-15T21:56:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.423+0000 7fa686b11640 1 -- 172.21.15.47:0/3440944395 shutdown_connections 2024-09-15T21:56:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.423+0000 7fa686b11640 1 --2- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa68012c4f0 0x7fa680136490 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.423+0000 7fa686b11640 1 --2- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fa680097550 0x7fa680135f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.423+0000 7fa686b11640 1 --2- 172.21.15.47:0/3440944395 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fa68006fcb0 0x7fa680135a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.423+0000 7fa686b11640 1 -- 172.21.15.47:0/3440944395 >> 172.21.15.47:0/3440944395 conn(0x7fa68009aba0 msgr2=0x7fa680095c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.423+0000 7fa686b11640 1 -- 172.21.15.47:0/3440944395 shutdown_connections 2024-09-15T21:56:51.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.423+0000 7fa686b11640 1 -- 172.21.15.47:0/3440944395 wait complete. 2024-09-15T21:56:51.462 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs dump' 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.559+0000 7f7d21be5640 1 Processor -- start 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.559+0000 7f7d21be5640 1 -- start start 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.559+0000 7f7d21be5640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d1c06d130 0x7f7d1c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.559+0000 7f7d21be5640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c12f940 0x7f7d1c12fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.559+0000 7f7d21be5640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d1c06e250 con 0x7f7d1c06da50 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.559+0000 7f7d21be5640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7d1c06ec40 con 0x7f7d1c06d130 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.559+0000 7f7d21be5640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f7d1c097890 con 0x7f7d1c12f940 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.560+0000 7f7d1bfff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c12f940 0x7f7d1c12fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.560+0000 7f7d1b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7d1c06d130 0x7f7d1c06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.560+0000 7f7d1affd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7d1c06da50 0x7f7d1c12f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34280/0 (socket says 172.21.15.47:34280) 2024-09-15T21:56:51.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.560+0000 7f7d1affd640 1 -- 172.21.15.47:0/2629838576 learned_addr learned my addr 172.21.15.47:0/2629838576 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.560+0000 7f7d1b7fe640 1 -- 172.21.15.47:0/2629838576 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c12f940 msgr2=0x7f7d1c12fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.560+0000 7f7d1b7fe640 1 --2- 172.21.15.47:0/2629838576 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c12f940 0x7f7d1c12fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.560+0000 7f7d1b7fe640 1 -- 172.21.15.47:0/2629838576 >> v1:172.21.15.47:6789/0 conn(0x7f7d1c06da50 legacy=0x7f7d1c12f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.560+0000 7f7d1a7fc640 1 -- 172.21.15.47:0/2629838576 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367645597 0 0) 0x7f7d1c06e250 con 0x7f7d1c06da50 2024-09-15T21:56:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.560+0000 7f7d1b7fe640 1 -- 172.21.15.47:0/2629838576 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c06ded0 con 0x7f7d1c06d130 2024-09-15T21:56:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.560+0000 7f7d1bfff640 1 --2- 172.21.15.47:0/2629838576 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c12f940 0x7f7d1c12fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d1b7fe640 1 --2- 172.21.15.47:0/2629838576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 0x7f7d1c06d510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0097a0 tx=0x7f7d0c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cecc19ae3b728b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d1a7fc640 1 -- 172.21.15.47:0/2629838576 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c037070 con 0x7f7d1c06d130 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d1a7fc640 1 -- 172.21.15.47:0/2629838576 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0c02ae50 con 0x7f7d1c06d130 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d1a7fc640 1 -- 172.21.15.47:0/2629838576 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c032660 con 0x7f7d1c06d130 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d21be5640 1 -- 172.21.15.47:0/2629838576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 msgr2=0x7f7d1c06d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d21be5640 1 --2- 172.21.15.47:0/2629838576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 0x7f7d1c06d510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0097a0 tx=0x7f7d0c02a100 comp rx=0 tx=0).stop 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d21be5640 1 -- 172.21.15.47:0/2629838576 shutdown_connections 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d21be5640 1 --2- 172.21.15.47:0/2629838576 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c12f940 0x7f7d1c12fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d21be5640 1 --2- 172.21.15.47:0/2629838576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 0x7f7d1c06d510 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d21be5640 1 -- 172.21.15.47:0/2629838576 >> 172.21.15.47:0/2629838576 conn(0x7f7d1c09fde0 msgr2=0x7f7d1c0a01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d21be5640 1 -- 172.21.15.47:0/2629838576 shutdown_connections 2024-09-15T21:56:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.561+0000 7f7d21be5640 1 -- 172.21.15.47:0/2629838576 wait complete. 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.562+0000 7f7d21be5640 1 Processor -- start 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.562+0000 7f7d21be5640 1 -- start start 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d21be5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 0x7f7d1c124cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d21be5640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c06da50 0x7f7d1c125200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d21be5640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c12f940 0x7f7d1c125740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d21be5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c137960 con 0x7f7d1c06d130 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d21be5640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1c1377e0 con 0x7f7d1c12f940 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d21be5640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f7d1c137ae0 con 0x7f7d1c06da50 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 0x7f7d1c124cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1affd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c06da50 0x7f7d1c125200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 0x7f7d1c124cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45728/0 (socket says 172.21.15.47:45728) 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1b7fe640 1 -- 172.21.15.47:0/489086894 learned_addr learned my addr 172.21.15.47:0/489086894 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:51.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1bfff640 1 --2- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c12f940 0x7f7d1c125740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1b7fe640 1 -- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c06da50 msgr2=0x7f7d1c125200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1b7fe640 1 --2- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c06da50 0x7f7d1c125200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1b7fe640 1 -- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c12f940 msgr2=0x7f7d1c125740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1b7fe640 1 --2- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c12f940 0x7f7d1c125740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1b7fe640 1 -- 172.21.15.47:0/489086894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c126fb0 con 0x7f7d1c06d130 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.563+0000 7f7d1b7fe640 1 --2- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 0x7f7d1c124cc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c035040 tx=0x7f7d0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.564+0000 7f7d03fff640 1 -- 172.21.15.47:0/489086894 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c041070 con 0x7f7d1c06d130 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.564+0000 7f7d03fff640 1 -- 172.21.15.47:0/489086894 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0c03c460 con 0x7f7d1c06d130 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.564+0000 7f7d21be5640 1 -- 172.21.15.47:0/489086894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1c099f90 con 0x7f7d1c06d130 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.564+0000 7f7d03fff640 1 -- 172.21.15.47:0/489086894 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c037040 con 0x7f7d1c06d130 2024-09-15T21:56:51.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.564+0000 7f7d21be5640 1 -- 172.21.15.47:0/489086894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1c126770 con 0x7f7d1c06d130 2024-09-15T21:56:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.565+0000 7f7d03fff640 1 -- 172.21.15.47:0/489086894 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f7d0c03a070 con 0x7f7d1c06d130 2024-09-15T21:56:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.565+0000 7f7d21be5640 1 -- 172.21.15.47:0/489086894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1c06d510 con 0x7f7d1c06d130 2024-09-15T21:56:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.565+0000 7f7d03fff640 1 -- 172.21.15.47:0/489086894 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3023+0+0 (secure 0 0 0) 0x7f7d0c07da40 con 0x7f7d1c06d130 2024-09-15T21:56:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.569+0000 7f7d03fff640 1 -- 172.21.15.47:0/489086894 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f7d0c083020 con 0x7f7d1c06d130 2024-09-15T21:56:51.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.744+0000 7f7d21be5640 1 -- 172.21.15.47:0/489086894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7d1c069040 con 0x7f7d1c06d130 2024-09-15T21:56:51.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.744+0000 7f7d03fff640 1 -- 172.21.15.47:0/489086894 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 1 v1) ==== 75+0+396 (secure 0 0 0) 0x7f7d1c126fb0 con 0x7f7d1c06d130 2024-09-15T21:56:51.746 INFO:teuthology.orchestra.run.smithi047.stdout:e1 2024-09-15T21:56:51.746 INFO:teuthology.orchestra.run.smithi047.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-15T21:56:51.746 INFO:teuthology.orchestra.run.smithi047.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T21:56:51.746 INFO:teuthology.orchestra.run.smithi047.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-15T21:56:51.746 INFO:teuthology.orchestra.run.smithi047.stdout:legacy client fscid: -1 2024-09-15T21:56:51.746 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:51.746 INFO:teuthology.orchestra.run.smithi047.stdout:No filesystems configured 2024-09-15T21:56:51.747 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 1 2024-09-15T21:56:51.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.746+0000 7f7d21be5640 1 -- 172.21.15.47:0/489086894 >> v1:172.21.15.47:6800/1790418198 conn(0x7f7d08048000 legacy=0x7f7d0804a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.746+0000 7f7d21be5640 1 -- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 msgr2=0x7f7d1c124cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.746+0000 7f7d21be5640 1 --2- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 0x7f7d1c124cc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c035040 tx=0x7f7d0c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.746+0000 7f7d21be5640 1 -- 172.21.15.47:0/489086894 shutdown_connections 2024-09-15T21:56:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.746+0000 7f7d21be5640 1 --2- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7d1c12f940 0x7f7d1c125740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.746+0000 7f7d21be5640 1 --2- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7d1c06da50 0x7f7d1c125200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.746+0000 7f7d21be5640 1 --2- 172.21.15.47:0/489086894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d1c06d130 0x7f7d1c124cc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.746+0000 7f7d21be5640 1 -- 172.21.15.47:0/489086894 >> 172.21.15.47:0/489086894 conn(0x7f7d1c09fde0 msgr2=0x7f7d1c12f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.746+0000 7f7d21be5640 1 -- 172.21.15.47:0/489086894 shutdown_connections 2024-09-15T21:56:51.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.746+0000 7f7d21be5640 1 -- 172.21.15.47:0/489086894 wait complete. 2024-09-15T21:56:51.758 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release squid' 2024-09-15T21:56:51.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd1a205a640 1 Processor -- start 2024-09-15T21:56:51.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd1a205a640 1 -- start start 2024-09-15T21:56:51.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd1a205a640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd1a205a640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c12f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd1a205a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd19c12fc70 0x7fd19c130120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd1a205a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd19c06ed40 con 0x7fd19c12fc70 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd1a205a640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c130660 con 0x7fd19c06ce30 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd1a205a640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd19c06e8a0 con 0x7fd19c06d750 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd19affd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c12f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd19b7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd19bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd19c12fc70 0x7fd19c130120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd19affd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c12f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:41922/0 (socket says 172.21.15.47:41922) 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd19affd640 1 -- 172.21.15.47:0/3124295261 learned_addr learned my addr 172.21.15.47:0/3124295261 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.855+0000 7fd19bfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd19c12fc70 0x7fd19c130120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45738/0 (socket says 172.21.15.47:45738) 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.856+0000 7fd19bfff640 1 -- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 msgr2=0x7fd19c12f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:51.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.856+0000 7fd19bfff640 1 --2- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c12f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.856+0000 7fd19bfff640 1 -- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 msgr2=0x7fd19c06d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:51.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.856+0000 7fd19bfff640 1 --2- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c06d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.856+0000 7fd19bfff640 1 -- 172.21.15.47:0/3124295261 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c099ce0 con 0x7fd19c12fc70 2024-09-15T21:56:51.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.856+0000 7fd19b7fe640 1 --2- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c06d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:51.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.856+0000 7fd19affd640 1 --2- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c12f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:51.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.856+0000 7fd19bfff640 1 --2- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c12fc70 0x7fd19c130120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd194009dc0 tx=0x7fd19402a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ac07a9b5a69e76 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.856+0000 7fd19a7fc640 1 -- 172.21.15.47:0/3124295261 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194007740 con 0x7fd19c12fc70 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd19a7fc640 1 -- 172.21.15.47:0/3124295261 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1940078e0 con 0x7fd19c12fc70 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd19a7fc640 1 -- 172.21.15.47:0/3124295261 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194007cb0 con 0x7fd19c12fc70 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd1a205a640 1 -- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c12fc70 msgr2=0x7fd19c130120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd1a205a640 1 --2- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c12fc70 0x7fd19c130120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd194009dc0 tx=0x7fd19402a4d0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd1a205a640 1 -- 172.21.15.47:0/3124295261 shutdown_connections 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd1a205a640 1 --2- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c12fc70 0x7fd19c130120 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd1a205a640 1 --2- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c12f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd1a205a640 1 --2- 172.21.15.47:0/3124295261 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c06d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd1a205a640 1 -- 172.21.15.47:0/3124295261 >> 172.21.15.47:0/3124295261 conn(0x7fd19c09fe80 msgr2=0x7fd19c0a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd1a205a640 1 -- 172.21.15.47:0/3124295261 shutdown_connections 2024-09-15T21:56:51.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.857+0000 7fd1a205a640 1 -- 172.21.15.47:0/3124295261 wait complete. 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.858+0000 7fd1a205a640 1 Processor -- start 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.858+0000 7fd1a205a640 1 -- start start 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd1a205a640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c131680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd1a205a640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c131bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd1a205a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c12fc70 0x7fd19c132100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd1a205a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c099a20 con 0x7fd19c12fc70 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd1a205a640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c0998a0 con 0x7fd19c06ce30 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd1a205a640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd19c136a60 con 0x7fd19c06d750 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19b7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c131680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19b7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c131680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44598/0 (socket says 172.21.15.47:44598) 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19affd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c131bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c12fc70 0x7fd19c132100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:51.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19b7fe640 1 -- 172.21.15.47:0/2262383612 learned_addr learned my addr 172.21.15.47:0/2262383612 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19affd640 1 -- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 msgr2=0x7fd19c131680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19affd640 1 --2- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c131680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19affd640 1 -- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c12fc70 msgr2=0x7fd19c132100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19affd640 1 --2- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c12fc70 0x7fd19c132100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19affd640 1 -- 172.21.15.47:0/2262383612 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c134140 con 0x7fd19c06d750 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19b7fe640 1 --2- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c131680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd19affd640 1 --2- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c131bc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd19000e980 tx=0x7fd19000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd183fff640 1 -- 172.21.15.47:0/2262383612 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190017070 con 0x7fd19c06d750 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd183fff640 1 -- 172.21.15.47:0/2262383612 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd19000ca00 con 0x7fd19c06d750 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd1a205a640 1 -- 172.21.15.47:0/2262383612 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c134420 con 0x7fd19c06d750 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd183fff640 1 -- 172.21.15.47:0/2262383612 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19000cdf0 con 0x7fd19c06d750 2024-09-15T21:56:51.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.859+0000 7fd1a205a640 1 -- 172.21.15.47:0/2262383612 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c135190 con 0x7fd19c06d750 2024-09-15T21:56:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.861+0000 7fd183fff640 1 -- 172.21.15.47:0/2262383612 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fd19001a070 con 0x7fd19c06d750 2024-09-15T21:56:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.861+0000 7fd1a205a640 1 -- 172.21.15.47:0/2262383612 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19c068930 con 0x7fd19c06d750 2024-09-15T21:56:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.861+0000 7fd183fff640 1 -- 172.21.15.47:0/2262383612 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(28..28 src has 1..28) ==== 3023+0+0 (secure 0 0 0) 0x7fd19005dba0 con 0x7fd19c06d750 2024-09-15T21:56:51.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:51.865+0000 7fd183fff640 1 -- 172.21.15.47:0/2262383612 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fd19c068930 con 0x7fd19c06d750 2024-09-15T21:56:52.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.020+0000 7fd1a205a640 1 -- 172.21.15.47:0/2262383612 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd require-osd-release", "release": "squid"} v 0) -- 0x7fd19c068f90 con 0x7fd19c06d750 2024-09-15T21:56:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.734+0000 7fd183fff640 1 -- 172.21.15.47:0/2262383612 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd require-osd-release", "release": "squid"}]=0 v29) ==== 91+0+0 (secure 0 0 0) 0x7fd19005cc60 con 0x7fd19c06d750 2024-09-15T21:56:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.737+0000 7fd1a205a640 1 -- 172.21.15.47:0/2262383612 >> v1:172.21.15.47:6800/1790418198 conn(0x7fd188047fd0 legacy=0x7fd18804a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.737+0000 7fd1a205a640 1 -- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 msgr2=0x7fd19c131bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.737+0000 7fd1a205a640 1 --2- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c131bc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd19000e980 tx=0x7fd19000ee40 comp rx=0 tx=0).stop 2024-09-15T21:56:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.737+0000 7fd1a205a640 1 -- 172.21.15.47:0/2262383612 shutdown_connections 2024-09-15T21:56:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.737+0000 7fd1a205a640 1 --2- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd19c12fc70 0x7fd19c132100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.737+0000 7fd1a205a640 1 --2- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd19c06d750 0x7fd19c131bc0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd19000e980 tx=0x7fd19000ee40 comp rx=0 tx=0).stop 2024-09-15T21:56:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.737+0000 7fd1a205a640 1 --2- 172.21.15.47:0/2262383612 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd19c06ce30 0x7fd19c131680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.737+0000 7fd1a205a640 1 -- 172.21.15.47:0/2262383612 >> 172.21.15.47:0/2262383612 conn(0x7fd19c09fe80 msgr2=0x7fd19c12eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.737+0000 7fd1a205a640 1 -- 172.21.15.47:0/2262383612 shutdown_connections 2024-09-15T21:56:52.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.738+0000 7fd1a205a640 1 -- 172.21.15.47:0/2262383612 wait complete. 2024-09-15T21:56:52.753 DEBUG:teuthology.orchestra.run.smithi047:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done' 2024-09-15T21:56:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.856+0000 7ff7ba6d2640 1 Processor -- start 2024-09-15T21:56:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.856+0000 7ff7ba6d2640 1 -- start start 2024-09-15T21:56:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7ba6d2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7b406eb40 0x7ff7b406dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7ba6d2640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b4134900 0x7ff7b4136cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7ba6d2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b40996f0 con 0x7ff7b406d1d0 2024-09-15T21:56:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7ba6d2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b406e200 con 0x7ff7b406eb40 2024-09-15T21:56:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7ba6d2640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7ff7b4138240 con 0x7ff7b4134900 2024-09-15T21:56:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7abfff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7b406eb40 0x7ff7b406dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:52.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7b8c6f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b4134900 0x7ff7b4136cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7b3fff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff7b406d1d0 0x7ff7b406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34284/0 (socket says 172.21.15.47:34284) 2024-09-15T21:56:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7b3fff640 1 -- 172.21.15.47:0/4198083317 learned_addr learned my addr 172.21.15.47:0/4198083317 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7b8c6f640 1 -- 172.21.15.47:0/4198083317 >> v2:172.21.15.47:3300/0 conn(0x7ff7b406eb40 msgr2=0x7ff7b406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7b8c6f640 1 --2- 172.21.15.47:0/4198083317 >> v2:172.21.15.47:3300/0 conn(0x7ff7b406eb40 0x7ff7b406dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7b8c6f640 1 -- 172.21.15.47:0/4198083317 >> v1:172.21.15.47:6789/0 conn(0x7ff7b406d1d0 legacy=0x7ff7b406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7b37fe640 1 -- 172.21.15.47:0/4198083317 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3571496985 0 0) 0x7ff7b40996f0 con 0x7ff7b406d1d0 2024-09-15T21:56:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7b8c6f640 1 -- 172.21.15.47:0/4198083317 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b41383c0 con 0x7ff7b4134900 2024-09-15T21:56:52.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.857+0000 7ff7abfff640 1 --2- 172.21.15.47:0/4198083317 >> v2:172.21.15.47:3300/0 conn(0x7ff7b406eb40 0x7ff7b406dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:52.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.858+0000 7ff7b8c6f640 1 --2- 172.21.15.47:0/4198083317 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b4134900 0x7ff7b4136cf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009dc0 tx=0x7ff7ac02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=739913103f02b317 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.858+0000 7ff7b37fe640 1 -- 172.21.15.47:0/4198083317 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac007740 con 0x7ff7b4134900 2024-09-15T21:56:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.858+0000 7ff7b37fe640 1 -- 172.21.15.47:0/4198083317 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ac0078e0 con 0x7ff7b4134900 2024-09-15T21:56:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.858+0000 7ff7b37fe640 1 -- 172.21.15.47:0/4198083317 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac007cb0 con 0x7ff7b4134900 2024-09-15T21:56:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.859+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/4198083317 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b4134900 msgr2=0x7ff7b4136cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.859+0000 7ff7ba6d2640 1 --2- 172.21.15.47:0/4198083317 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b4134900 0x7ff7b4136cf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009dc0 tx=0x7ff7ac02a560 comp rx=0 tx=0).stop 2024-09-15T21:56:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.859+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/4198083317 shutdown_connections 2024-09-15T21:56:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.859+0000 7ff7ba6d2640 1 --2- 172.21.15.47:0/4198083317 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b4134900 0x7ff7b4136cf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.859+0000 7ff7ba6d2640 1 --2- 172.21.15.47:0/4198083317 >> v2:172.21.15.47:3300/0 conn(0x7ff7b406eb40 0x7ff7b406dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.859+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/4198083317 >> 172.21.15.47:0/4198083317 conn(0x7ff7b409ff10 msgr2=0x7ff7b40a0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:52.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.859+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/4198083317 shutdown_connections 2024-09-15T21:56:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.859+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/4198083317 wait complete. 2024-09-15T21:56:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7ba6d2640 1 Processor -- start 2024-09-15T21:56:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7ba6d2640 1 -- start start 2024-09-15T21:56:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7ba6d2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b406d1d0 0x7ff7b412d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7ba6d2640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b406eb40 0x7ff7b412dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7ba6d2640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7b4134900 0x7ff7b41c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7ba6d2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41375c0 con 0x7ff7b406d1d0 2024-09-15T21:56:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7ba6d2640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b4137440 con 0x7ff7b4134900 2024-09-15T21:56:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7ba6d2640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7ff7b4137740 con 0x7ff7b406eb40 2024-09-15T21:56:52.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7b3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b406d1d0 0x7ff7b412d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7abfff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b406eb40 0x7ff7b412dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7abfff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b406eb40 0x7ff7b412dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:41948/0 (socket says 172.21.15.47:41948) 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7abfff640 1 -- 172.21.15.47:0/2311901110 learned_addr learned my addr 172.21.15.47:0/2311901110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7b3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b406d1d0 0x7ff7b412d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45766/0 (socket says 172.21.15.47:45766) 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7b8c6f640 1 --2- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7b4134900 0x7ff7b41c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7abfff640 1 -- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7b4134900 msgr2=0x7ff7b41c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7abfff640 1 --2- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7b4134900 0x7ff7b41c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7abfff640 1 -- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b406d1d0 msgr2=0x7ff7b412d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7abfff640 1 --2- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b406d1d0 0x7ff7b412d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7abfff640 1 -- 172.21.15.47:0/2311901110 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b4130bc0 con 0x7ff7b406eb40 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.860+0000 7ff7b3fff640 1 --2- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b406d1d0 0x7ff7b412d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.861+0000 7ff7abfff640 1 --2- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b406eb40 0x7ff7b412dc80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00ebe0 tx=0x7ff79c00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:52.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.861+0000 7ff7b17fa640 1 -- 172.21.15.47:0/2311901110 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c017070 con 0x7ff7b406eb40 2024-09-15T21:56:52.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.861+0000 7ff7b17fa640 1 -- 172.21.15.47:0/2311901110 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c00cc30 con 0x7ff7b406eb40 2024-09-15T21:56:52.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.861+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/2311901110 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b4130e60 con 0x7ff7b406eb40 2024-09-15T21:56:52.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.861+0000 7ff7b17fa640 1 -- 172.21.15.47:0/2311901110 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c013470 con 0x7ff7b406eb40 2024-09-15T21:56:52.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.861+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/2311901110 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b412e1c0 con 0x7ff7b406eb40 2024-09-15T21:56:52.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.862+0000 7ff7b17fa640 1 -- 172.21.15.47:0/2311901110 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7ff79c01a070 con 0x7ff7b406eb40 2024-09-15T21:56:52.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.862+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/2311901110 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b40a2d40 con 0x7ff7b406eb40 2024-09-15T21:56:52.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.862+0000 7ff7b17fa640 1 -- 172.21.15.47:0/2311901110 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(29..29 src has 1..29) ==== 3027+0+0 (secure 0 0 0) 0x7ff79c05d8d0 con 0x7ff7b406eb40 2024-09-15T21:56:52.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:52.866+0000 7ff7b17fa640 1 -- 172.21.15.47:0/2311901110 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7ff7b40a2d40 con 0x7ff7b406eb40 2024-09-15T21:56:53.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.013+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/2311901110 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd pool ls"} v 0) -- 0x7ff7b4137b20 con 0x7ff7b406eb40 2024-09-15T21:56:53.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.014+0000 7ff7b17fa640 1 -- 172.21.15.47:0/2311901110 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v29) ==== 59+0+9 (secure 0 0 0) 0x7ff7b4130bc0 con 0x7ff7b406eb40 2024-09-15T21:56:53.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.015+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/2311901110 >> v1:172.21.15.47:6800/1790418198 conn(0x7ff794047fd0 legacy=0x7ff79404a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.015+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b406eb40 msgr2=0x7ff7b412dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.015+0000 7ff7ba6d2640 1 --2- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b406eb40 0x7ff7b412dc80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00ebe0 tx=0x7ff79c00c520 comp rx=0 tx=0).stop 2024-09-15T21:56:53.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.015+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/2311901110 shutdown_connections 2024-09-15T21:56:53.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.015+0000 7ff7ba6d2640 1 --2- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff7b4134900 0x7ff7b41c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.015+0000 7ff7ba6d2640 1 --2- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff7b406eb40 0x7ff7b412dc80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.015+0000 7ff7ba6d2640 1 --2- 172.21.15.47:0/2311901110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7b406d1d0 0x7ff7b412d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.015+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/2311901110 >> 172.21.15.47:0/2311901110 conn(0x7ff7b409ff10 msgr2=0x7ff7b409d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:53.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.015+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/2311901110 shutdown_connections 2024-09-15T21:56:53.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.015+0000 7ff7ba6d2640 1 -- 172.21.15.47:0/2311901110 wait complete. 2024-09-15T21:56:53.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.102+0000 7f6b9c248640 1 Processor -- start 2024-09-15T21:56:53.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.103+0000 7f6b9c248640 1 -- start start 2024-09-15T21:56:53.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.103+0000 7f6b9c248640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b9412f4b0 0x7f6b9412f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.103+0000 7f6b9c248640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b94098790 0x7f6b94137a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.103+0000 7f6b9c248640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b940a1510 con 0x7f6b9412fdd0 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.103+0000 7f6b9c248640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6b94099790 con 0x7f6b9412f4b0 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.103+0000 7f6b9c248640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f6b9409dd90 con 0x7f6b94098790 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.103+0000 7f6b9a7e5640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b94098790 0x7f6b94137a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.103+0000 7f6b99fe4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b9412f4b0 0x7f6b9412f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.103+0000 7f6b997e3640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6b9412fdd0 0x7f6b94098080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34300/0 (socket says 172.21.15.47:34300) 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.103+0000 7f6b997e3640 1 -- 172.21.15.47:0/3065403668 learned_addr learned my addr 172.21.15.47:0/3065403668 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.104+0000 7f6b9a7e5640 1 -- 172.21.15.47:0/3065403668 >> v2:172.21.15.47:3300/0 conn(0x7f6b9412f4b0 msgr2=0x7f6b9412f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.104+0000 7f6b9a7e5640 1 --2- 172.21.15.47:0/3065403668 >> v2:172.21.15.47:3300/0 conn(0x7f6b9412f4b0 0x7f6b9412f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.104+0000 7f6b9a7e5640 1 -- 172.21.15.47:0/3065403668 >> v1:172.21.15.47:6789/0 conn(0x7f6b9412fdd0 legacy=0x7f6b94098080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.104+0000 7f6b9a7e5640 1 -- 172.21.15.47:0/3065403668 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9409db20 con 0x7f6b94098790 2024-09-15T21:56:53.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.104+0000 7f6b99fe4640 1 --2- 172.21.15.47:0/3065403668 >> v2:172.21.15.47:3300/0 conn(0x7f6b9412f4b0 0x7f6b9412f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:53.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.104+0000 7f6b9a7e5640 1 --2- 172.21.15.47:0/3065403668 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b94098790 0x7f6b94137a70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009d90 tx=0x7f6b8c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b123c5d202946798 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:53.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.104+0000 7f6b98fe2640 1 -- 172.21.15.47:0/3065403668 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c031600 con 0x7f6b94098790 2024-09-15T21:56:53.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.104+0000 7f6b98fe2640 1 -- 172.21.15.47:0/3065403668 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8c0317a0 con 0x7f6b94098790 2024-09-15T21:56:53.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.104+0000 7f6b98fe2640 1 -- 172.21.15.47:0/3065403668 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c031b70 con 0x7f6b94098790 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.105+0000 7f6b9c248640 1 -- 172.21.15.47:0/3065403668 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b94098790 msgr2=0x7f6b94137a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.105+0000 7f6b9c248640 1 --2- 172.21.15.47:0/3065403668 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b94098790 0x7f6b94137a70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009d90 tx=0x7f6b8c02a5d0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.105+0000 7f6b9c248640 1 -- 172.21.15.47:0/3065403668 shutdown_connections 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.105+0000 7f6b9c248640 1 --2- 172.21.15.47:0/3065403668 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b94098790 0x7f6b94137a70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.105+0000 7f6b9c248640 1 --2- 172.21.15.47:0/3065403668 >> v2:172.21.15.47:3300/0 conn(0x7f6b9412f4b0 0x7f6b9412f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.105+0000 7f6b9c248640 1 -- 172.21.15.47:0/3065403668 >> 172.21.15.47:0/3065403668 conn(0x7f6b9409edc0 msgr2=0x7f6b9409f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.105+0000 7f6b9c248640 1 -- 172.21.15.47:0/3065403668 shutdown_connections 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.105+0000 7f6b9c248640 1 -- 172.21.15.47:0/3065403668 wait complete. 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b9c248640 1 Processor -- start 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b9c248640 1 -- start start 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b9c248640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b94098790 0x7f6b941c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b9c248640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b9412f4b0 0x7f6b941c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b9c248640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b9412fdd0 0x7f6b9408cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b9c248640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94098bc0 con 0x7f6b9412fdd0 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b9c248640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f6b94130200 con 0x7f6b94098790 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b9c248640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f6b94098d40 con 0x7f6b9412f4b0 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b9a7e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b9412fdd0 0x7f6b9408cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b997e3640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b9412f4b0 0x7f6b941c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b99fe4640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b94098790 0x7f6b941c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.106+0000 7f6b9a7e5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b9412fdd0 0x7f6b9408cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45774/0 (socket says 172.21.15.47:45774) 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b9a7e5640 1 -- 172.21.15.47:0/321505790 learned_addr learned my addr 172.21.15.47:0/321505790 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b9a7e5640 1 -- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b9412f4b0 msgr2=0x7f6b941c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b9a7e5640 1 --2- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b9412f4b0 0x7f6b941c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b9a7e5640 1 -- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b94098790 msgr2=0x7f6b941c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b9a7e5640 1 --2- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b94098790 0x7f6b941c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b9a7e5640 1 -- 172.21.15.47:0/321505790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9408ef90 con 0x7f6b9412fdd0 2024-09-15T21:56:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b997e3640 1 --2- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b9412f4b0 0x7f6b941c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b9a7e5640 1 --2- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b9412fdd0 0x7f6b9408cfd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009eb0 tx=0x7f6b8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b8a7fc640 1 -- 172.21.15.47:0/321505790 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c040070 con 0x7f6b9412fdd0 2024-09-15T21:56:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b8a7fc640 1 -- 172.21.15.47:0/321505790 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8c030b60 con 0x7f6b9412fdd0 2024-09-15T21:56:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b8a7fc640 1 -- 172.21.15.47:0/321505790 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c009450 con 0x7f6b9412fdd0 2024-09-15T21:56:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b9c248640 1 -- 172.21.15.47:0/321505790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9409c280 con 0x7f6b9412fdd0 2024-09-15T21:56:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.107+0000 7f6b9c248640 1 -- 172.21.15.47:0/321505790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9408fc50 con 0x7f6b9412fdd0 2024-09-15T21:56:53.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.108+0000 7f6b8a7fc640 1 -- 172.21.15.47:0/321505790 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f6b8c039040 con 0x7f6b9412fdd0 2024-09-15T21:56:53.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.108+0000 7f6b9c248640 1 -- 172.21.15.47:0/321505790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9406d690 con 0x7f6b9412fdd0 2024-09-15T21:56:53.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.109+0000 7f6b8a7fc640 1 -- 172.21.15.47:0/321505790 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3027+0+0 (secure 0 0 0) 0x7f6b8c07ef40 con 0x7f6b9412fdd0 2024-09-15T21:56:53.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.112+0000 7f6b8a7fc640 1 -- 172.21.15.47:0/321505790 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f6b9406d690 con 0x7f6b9412fdd0 2024-09-15T21:56:53.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.263+0000 7f6b9c248640 1 -- 172.21.15.47:0/321505790 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"} v 0) -- 0x7f6b9412f890 con 0x7f6b9412fdd0 2024-09-15T21:56:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.738+0000 7f6b8a7fc640 1 -- 172.21.15.47:0/321505790 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 1 pg_autoscale_mode to off v30) ==== 153+0+0 (secure 0 0 0) 0x7f6b8c07dfd0 con 0x7f6b9412fdd0 2024-09-15T21:56:53.741 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 1 pg_autoscale_mode to off 2024-09-15T21:56:53.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.742+0000 7f6b9c248640 1 -- 172.21.15.47:0/321505790 >> v1:172.21.15.47:6800/1790418198 conn(0x7f6b78047fd0 legacy=0x7f6b7804a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.742+0000 7f6b9c248640 1 -- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b9412fdd0 msgr2=0x7f6b9408cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.742+0000 7f6b9c248640 1 --2- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b9412fdd0 0x7f6b9408cfd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009eb0 tx=0x7f6b8c004a40 comp rx=0 tx=0).stop 2024-09-15T21:56:53.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.742+0000 7f6b9c248640 1 -- 172.21.15.47:0/321505790 shutdown_connections 2024-09-15T21:56:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.742+0000 7f6b9c248640 1 --2- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b9412fdd0 0x7f6b9408cfd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.742+0000 7f6b9c248640 1 --2- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b9412f4b0 0x7f6b941c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.742+0000 7f6b9c248640 1 --2- 172.21.15.47:0/321505790 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b94098790 0x7f6b941c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.742+0000 7f6b9c248640 1 -- 172.21.15.47:0/321505790 >> 172.21.15.47:0/321505790 conn(0x7f6b9409edc0 msgr2=0x7f6b9409a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.742+0000 7f6b9c248640 1 -- 172.21.15.47:0/321505790 shutdown_connections 2024-09-15T21:56:53.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.742+0000 7f6b9c248640 1 -- 172.21.15.47:0/321505790 wait complete. 2024-09-15T21:56:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.834+0000 7f30fd92c640 1 Processor -- start 2024-09-15T21:56:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30fd92c640 1 -- start start 2024-09-15T21:56:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30fd92c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 0x7f30f806de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30fd92c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 0x7f30f806e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30fd92c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30f812fbe0 0x7f30f8131fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30fd92c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f30f806e910 con 0x7f30f812fbe0 2024-09-15T21:56:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30fd92c640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f30f8068590 con 0x7f30f806f4e0 2024-09-15T21:56:53.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30fd92c640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f30f80698f0 con 0x7f30f806dab0 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30f77fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30f812fbe0 0x7f30f8131fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30ee7fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 0x7f30f806e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30f77fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f30f812fbe0 0x7f30f8131fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45784/0 (socket says 172.21.15.47:45784) 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30f77fe640 1 -- 172.21.15.47:0/3953182368 learned_addr learned my addr 172.21.15.47:0/3953182368 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30f6ffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 0x7f30f806de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30f77fe640 1 -- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 msgr2=0x7f30f806de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30f77fe640 1 --2- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 0x7f30f806de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30f77fe640 1 -- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 msgr2=0x7f30f806e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30f77fe640 1 --2- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 0x7f30f806e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30f77fe640 1 -- 172.21.15.47:0/3953182368 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f806d220 con 0x7f30f812fbe0 2024-09-15T21:56:53.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.835+0000 7f30ee7fc640 1 --2- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 0x7f30f806e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.836+0000 7f30f77fe640 1 --2- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30f812fbe0 0x7f30f8131fd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009eb0 tx=0x7f30f002a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c584ac8cfca962da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.836+0000 7f30f67fc640 1 -- 172.21.15.47:0/3953182368 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0036070 con 0x7f30f812fbe0 2024-09-15T21:56:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.836+0000 7f30f67fc640 1 -- 172.21.15.47:0/3953182368 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f0031600 con 0x7f30f812fbe0 2024-09-15T21:56:53.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.836+0000 7f30f67fc640 1 -- 172.21.15.47:0/3953182368 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f00319d0 con 0x7f30f812fbe0 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.837+0000 7f30fd92c640 1 -- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30f812fbe0 msgr2=0x7f30f8131fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.837+0000 7f30fd92c640 1 --2- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30f812fbe0 0x7f30f8131fd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009eb0 tx=0x7f30f002a5b0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.837+0000 7f30fd92c640 1 -- 172.21.15.47:0/3953182368 shutdown_connections 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.837+0000 7f30fd92c640 1 --2- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30f812fbe0 0x7f30f8131fd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.837+0000 7f30fd92c640 1 --2- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 0x7f30f806e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.837+0000 7f30fd92c640 1 --2- 172.21.15.47:0/3953182368 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 0x7f30f806de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.837+0000 7f30fd92c640 1 -- 172.21.15.47:0/3953182368 >> 172.21.15.47:0/3953182368 conn(0x7f30f809bd90 msgr2=0x7f30f809c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.837+0000 7f30fd92c640 1 -- 172.21.15.47:0/3953182368 shutdown_connections 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.837+0000 7f30fd92c640 1 -- 172.21.15.47:0/3953182368 wait complete. 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.838+0000 7f30fd92c640 1 Processor -- start 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.838+0000 7f30fd92c640 1 -- start start 2024-09-15T21:56:53.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.838+0000 7f30fd92c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 0x7f30f8122b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.838+0000 7f30fd92c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 0x7f30f8123070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.838+0000 7f30fd92c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30f812fbe0 0x7f30f81235b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.838+0000 7f30fd92c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f30f8133620 con 0x7f30f812fbe0 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.838+0000 7f30fd92c640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f30f806d800 con 0x7f30f806f4e0 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.838+0000 7f30fd92c640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f30f81337a0 con 0x7f30f806dab0 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.838+0000 7f30f6ffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 0x7f30f8122b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30ee7fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 0x7f30f8123070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f6ffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 0x7f30f8122b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:41968/0 (socket says 172.21.15.47:41968) 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f6ffd640 1 -- 172.21.15.47:0/1243666110 learned_addr learned my addr 172.21.15.47:0/1243666110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f77fe640 1 --2- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30f812fbe0 0x7f30f81235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f77fe640 1 -- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 msgr2=0x7f30f8122b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f77fe640 1 --2- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 0x7f30f8122b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f77fe640 1 -- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 msgr2=0x7f30f8123070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:53.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f77fe640 1 --2- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 0x7f30f8123070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f77fe640 1 -- 172.21.15.47:0/1243666110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f8072740 con 0x7f30f812fbe0 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f6ffd640 1 --2- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 0x7f30f8122b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30ee7fc640 1 --2- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 0x7f30f8123070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f77fe640 1 --2- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30f812fbe0 0x7f30f81235b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f30f0030000 tx=0x7f30f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f4ff9640 1 -- 172.21.15.47:0/1243666110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0036040 con 0x7f30f812fbe0 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f4ff9640 1 -- 172.21.15.47:0/1243666110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f0003990 con 0x7f30f812fbe0 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30fd92c640 1 -- 172.21.15.47:0/1243666110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f8124300 con 0x7f30f812fbe0 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30f4ff9640 1 -- 172.21.15.47:0/1243666110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0003da0 con 0x7f30f812fbe0 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.839+0000 7f30fd92c640 1 -- 172.21.15.47:0/1243666110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f8095f90 con 0x7f30f812fbe0 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.840+0000 7f30fd92c640 1 -- 172.21.15.47:0/1243666110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f8096220 con 0x7f30f812fbe0 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.840+0000 7f30f4ff9640 1 -- 172.21.15.47:0/1243666110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f30f00077a0 con 0x7f30f812fbe0 2024-09-15T21:56:53.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.841+0000 7f30f4ff9640 1 -- 172.21.15.47:0/1243666110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3027+0+0 (secure 0 0 0) 0x7f30f007e930 con 0x7f30f812fbe0 2024-09-15T21:56:53.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.844+0000 7f30f4ff9640 1 -- 172.21.15.47:0/1243666110 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f30f8096220 con 0x7f30f812fbe0 2024-09-15T21:56:53.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:53.991+0000 7f30fd92c640 1 -- 172.21.15.47:0/1243666110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"} v 0) -- 0x7f30f8099f40 con 0x7f30f812fbe0 2024-09-15T21:56:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.739+0000 7f30f4ff9640 1 -- 172.21.15.47:0/1243666110 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 2 pg_autoscale_mode to off v31) ==== 152+0+0 (secure 0 0 0) 0x7f30f8099f40 con 0x7f30f812fbe0 2024-09-15T21:56:54.742 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 2 pg_autoscale_mode to off 2024-09-15T21:56:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.743+0000 7f30fd92c640 1 -- 172.21.15.47:0/1243666110 >> v1:172.21.15.47:6800/1790418198 conn(0x7f30d8047fd0 legacy=0x7f30d804a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.743+0000 7f30fd92c640 1 -- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30f812fbe0 msgr2=0x7f30f81235b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.743+0000 7f30fd92c640 1 --2- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30f812fbe0 0x7f30f81235b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f30f0030000 tx=0x7f30f0004a40 comp rx=0 tx=0).stop 2024-09-15T21:56:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.743+0000 7f30fd92c640 1 -- 172.21.15.47:0/1243666110 shutdown_connections 2024-09-15T21:56:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.743+0000 7f30fd92c640 1 --2- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f30f812fbe0 0x7f30f81235b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.743+0000 7f30fd92c640 1 --2- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f30f806f4e0 0x7f30f8123070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.743+0000 7f30fd92c640 1 --2- 172.21.15.47:0/1243666110 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f30f806dab0 0x7f30f8122b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.743+0000 7f30fd92c640 1 -- 172.21.15.47:0/1243666110 >> 172.21.15.47:0/1243666110 conn(0x7f30f809bd90 msgr2=0x7f30f8097650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.743+0000 7f30fd92c640 1 -- 172.21.15.47:0/1243666110 shutdown_connections 2024-09-15T21:56:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.743+0000 7f30fd92c640 1 -- 172.21.15.47:0/1243666110 wait complete. 2024-09-15T21:56:54.760 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-09-15T21:56:54.774 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T21:56:54.774 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T21:56:54.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T21:56:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.881+0000 7f65a6861640 1 Processor -- start 2024-09-15T21:56:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.881+0000 7f65a6861640 1 -- start start 2024-09-15T21:56:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.881+0000 7f65a6861640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012f6f0 0x7f65a012fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.881+0000 7f65a6861640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a01303d0 0x7f65a012c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f65a6861640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65a012c730 0x7f65a012cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f65a6861640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f65a0098630 con 0x7f65a012c730 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f65a6861640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f65a0091820 con 0x7f65a012f6f0 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f65a6861640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f65a0091ad0 con 0x7f65a01303d0 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f659f7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a01303d0 0x7f65a012c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f659ffff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012f6f0 0x7f65a012fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f65a4dfe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65a012c730 0x7f65a012cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f659f7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a01303d0 0x7f65a012c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:41970/0 (socket says 172.21.15.47:41970) 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f659ffff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012f6f0 0x7f65a012fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44644/0 (socket says 172.21.15.47:44644) 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f659f7fe640 1 -- 172.21.15.47:0/18715529 learned_addr learned my addr 172.21.15.47:0/18715529 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f659f7fe640 1 -- 172.21.15.47:0/18715529 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012f6f0 msgr2=0x7f65a012fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f659f7fe640 1 --2- 172.21.15.47:0/18715529 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012f6f0 0x7f65a012fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f659f7fe640 1 -- 172.21.15.47:0/18715529 >> v2:172.21.15.47:3300/0 conn(0x7f65a012c730 msgr2=0x7f65a012cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f659f7fe640 1 --2- 172.21.15.47:0/18715529 >> v2:172.21.15.47:3300/0 conn(0x7f65a012c730 0x7f65a012cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f659f7fe640 1 -- 172.21.15.47:0/18715529 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a0099090 con 0x7f65a01303d0 2024-09-15T21:56:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.882+0000 7f65a4dfe640 1 --2- 172.21.15.47:0/18715529 >> v2:172.21.15.47:3300/0 conn(0x7f65a012c730 0x7f65a012cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.883+0000 7f659f7fe640 1 --2- 172.21.15.47:0/18715529 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a01303d0 0x7f65a012c1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6594009870 tx=0x7f659402a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef55ac0c85d1e8bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.883+0000 7f659effd640 1 -- 172.21.15.47:0/18715529 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594037070 con 0x7f65a01303d0 2024-09-15T21:56:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.883+0000 7f659effd640 1 -- 172.21.15.47:0/18715529 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659402adf0 con 0x7f65a01303d0 2024-09-15T21:56:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.883+0000 7f659effd640 1 -- 172.21.15.47:0/18715529 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594032660 con 0x7f65a01303d0 2024-09-15T21:56:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.884+0000 7f65a6861640 1 -- 172.21.15.47:0/18715529 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a01303d0 msgr2=0x7f65a012c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.884+0000 7f65a6861640 1 --2- 172.21.15.47:0/18715529 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a01303d0 0x7f65a012c1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6594009870 tx=0x7f659402a0a0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.884+0000 7f65a6861640 1 -- 172.21.15.47:0/18715529 shutdown_connections 2024-09-15T21:56:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.884+0000 7f65a6861640 1 --2- 172.21.15.47:0/18715529 >> v2:172.21.15.47:3300/0 conn(0x7f65a012c730 0x7f65a012cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.884+0000 7f65a6861640 1 --2- 172.21.15.47:0/18715529 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a01303d0 0x7f65a012c1f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.884+0000 7f65a6861640 1 --2- 172.21.15.47:0/18715529 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012f6f0 0x7f65a012fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.884+0000 7f65a6861640 1 -- 172.21.15.47:0/18715529 >> 172.21.15.47:0/18715529 conn(0x7f65a009b770 msgr2=0x7f65a009bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.884+0000 7f65a6861640 1 -- 172.21.15.47:0/18715529 shutdown_connections 2024-09-15T21:56:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.884+0000 7f65a6861640 1 -- 172.21.15.47:0/18715529 wait complete. 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.884+0000 7f65a6861640 1 Processor -- start 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f65a6861640 1 -- start start 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f65a6861640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012c730 0x7f65a01321f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f65a6861640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a012f6f0 0x7f65a0132730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f65a6861640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a01303d0 0x7f65a0132c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f65a6861640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65a0136ba0 con 0x7f65a01303d0 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f65a6861640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f65a0136a20 con 0x7f65a012c730 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f65a6861640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f65a0136d20 con 0x7f65a012f6f0 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f659f7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a012f6f0 0x7f65a0132730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f659ffff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012c730 0x7f65a01321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f65a4dfe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a01303d0 0x7f65a0132c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f659f7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a012f6f0 0x7f65a0132730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:41984/0 (socket says 172.21.15.47:41984) 2024-09-15T21:56:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.885+0000 7f659f7fe640 1 -- 172.21.15.47:0/1042461405 learned_addr learned my addr 172.21.15.47:0/1042461405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f659f7fe640 1 -- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012c730 msgr2=0x7f65a01321f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f659f7fe640 1 --2- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012c730 0x7f65a01321f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f659f7fe640 1 -- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a01303d0 msgr2=0x7f65a0132c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f659f7fe640 1 --2- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a01303d0 0x7f65a0132c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f659f7fe640 1 -- 172.21.15.47:0/1042461405 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a01344f0 con 0x7f65a012f6f0 2024-09-15T21:56:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f65a4dfe640 1 --2- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a01303d0 0x7f65a0132c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f659f7fe640 1 --2- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a012f6f0 0x7f65a0132730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f659402a0e0 tx=0x7f65940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f659cff9640 1 -- 172.21.15.47:0/1042461405 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594041070 con 0x7f65a012f6f0 2024-09-15T21:56:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f659cff9640 1 -- 172.21.15.47:0/1042461405 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659403c460 con 0x7f65a012f6f0 2024-09-15T21:56:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f65a6861640 1 -- 172.21.15.47:0/1042461405 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a01364c0 con 0x7f65a012f6f0 2024-09-15T21:56:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f659cff9640 1 -- 172.21.15.47:0/1042461405 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594037040 con 0x7f65a012f6f0 2024-09-15T21:56:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.886+0000 7f65a6861640 1 -- 172.21.15.47:0/1042461405 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a0134c40 con 0x7f65a012f6f0 2024-09-15T21:56:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.887+0000 7f65a6861640 1 -- 172.21.15.47:0/1042461405 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a006c160 con 0x7f65a012f6f0 2024-09-15T21:56:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.887+0000 7f659cff9640 1 -- 172.21.15.47:0/1042461405 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f6594044070 con 0x7f65a012f6f0 2024-09-15T21:56:54.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.888+0000 7f659cff9640 1 -- 172.21.15.47:0/1042461405 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f659407de90 con 0x7f65a012f6f0 2024-09-15T21:56:54.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:54.891+0000 7f659cff9640 1 -- 172.21.15.47:0/1042461405 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f65a006c160 con 0x7f65a012f6f0 2024-09-15T21:56:55.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.069+0000 7f65a6861640 1 -- 172.21.15.47:0/1042461405 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f65a012fe90 con 0x7f65a012f6f0 2024-09-15T21:56:55.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.071+0000 7f659cff9640 1 -- 172.21.15.47:0/1042461405 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v10) ==== 74+0+108903 (secure 0 0 0) 0x7f659407cf40 con 0x7f65a012f6f0 2024-09-15T21:56:55.075 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.073+0000 7f65a6861640 1 -- 172.21.15.47:0/1042461405 >> v1:172.21.15.47:6800/1790418198 conn(0x7f658c047fd0 legacy=0x7f658c04a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.073+0000 7f65a6861640 1 -- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a012f6f0 msgr2=0x7f65a0132730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.073+0000 7f65a6861640 1 --2- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a012f6f0 0x7f65a0132730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f659402a0e0 tx=0x7f65940046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.073+0000 7f65a6861640 1 -- 172.21.15.47:0/1042461405 shutdown_connections 2024-09-15T21:56:55.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.073+0000 7f65a6861640 1 --2- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a01303d0 0x7f65a0132c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.073+0000 7f65a6861640 1 --2- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f65a012f6f0 0x7f65a0132730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.073+0000 7f65a6861640 1 --2- 172.21.15.47:0/1042461405 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f65a012c730 0x7f65a01321f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.073+0000 7f65a6861640 1 -- 172.21.15.47:0/1042461405 >> 172.21.15.47:0/1042461405 conn(0x7f65a009b770 msgr2=0x7f65a012ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.073+0000 7f65a6861640 1 -- 172.21.15.47:0/1042461405 shutdown_connections 2024-09-15T21:56:55.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.073+0000 7f65a6861640 1 -- 172.21.15.47:0/1042461405 wait complete. 2024-09-15T21:56:55.087 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"flags":0,"active_gid":14098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6800","nonce":1790418198}]},"active_addr":"172.21.15.47:6800/1790418198","active_change":"2024-09-15T21:56:33.662052+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":14105,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":19,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":185820581}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":885911555}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3434580097}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":2167288974}]}]} 2024-09-15T21:56:55.090 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T21:56:55.091 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:56:55.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:56:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe726d33640 1 Processor -- start 2024-09-15T21:56:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe726d33640 1 -- start start 2024-09-15T21:56:55.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe726d33640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720095e60 0x7fe720132670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe726d33640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720132bb0 0x7fe720134fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe726d33640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe720097580 con 0x7fe720095370 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe726d33640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fe720097c80 con 0x7fe720095e60 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe726d33640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fe7200979c0 con 0x7fe720132bb0 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe724acf640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe720095370 0x7fe720095750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34314/0 (socket says 172.21.15.47:34314) 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe724acf640 1 -- 172.21.15.47:0/166876171 learned_addr learned my addr 172.21.15.47:0/166876171 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe7252d0640 1 --2- 172.21.15.47:0/166876171 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720132bb0 0x7fe720134fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.190+0000 7fe71ffff640 1 --2- 172.21.15.47:0/166876171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720095e60 0x7fe720132670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.191+0000 7fe71f7fe640 1 -- 172.21.15.47:0/166876171 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1250467581 0 0) 0x7fe720097580 con 0x7fe720095370 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.191+0000 7fe7252d0640 1 -- 172.21.15.47:0/166876171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720095e60 msgr2=0x7fe720132670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.191+0000 7fe7252d0640 1 --2- 172.21.15.47:0/166876171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720095e60 0x7fe720132670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.191+0000 7fe7252d0640 1 -- 172.21.15.47:0/166876171 >> v1:172.21.15.47:6789/0 conn(0x7fe720095370 legacy=0x7fe720095750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.191+0000 7fe7252d0640 1 -- 172.21.15.47:0/166876171 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7200750e0 con 0x7fe720132bb0 2024-09-15T21:56:55.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.191+0000 7fe71ffff640 1 --2- 172.21.15.47:0/166876171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720095e60 0x7fe720132670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.191+0000 7fe7252d0640 1 --2- 172.21.15.47:0/166876171 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720132bb0 0x7fe720134fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe718009dc0 tx=0x7fe71802a510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb339fe907c44828 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.191+0000 7fe71f7fe640 1 -- 172.21.15.47:0/166876171 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718007740 con 0x7fe720132bb0 2024-09-15T21:56:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.192+0000 7fe71f7fe640 1 -- 172.21.15.47:0/166876171 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7180078e0 con 0x7fe720132bb0 2024-09-15T21:56:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.192+0000 7fe71f7fe640 1 -- 172.21.15.47:0/166876171 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718007cb0 con 0x7fe720132bb0 2024-09-15T21:56:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.192+0000 7fe726d33640 1 -- 172.21.15.47:0/166876171 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720132bb0 msgr2=0x7fe720134fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.192+0000 7fe726d33640 1 --2- 172.21.15.47:0/166876171 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720132bb0 0x7fe720134fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe718009dc0 tx=0x7fe71802a510 comp rx=0 tx=0).stop 2024-09-15T21:56:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.192+0000 7fe726d33640 1 -- 172.21.15.47:0/166876171 shutdown_connections 2024-09-15T21:56:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.192+0000 7fe726d33640 1 --2- 172.21.15.47:0/166876171 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720132bb0 0x7fe720134fa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.192+0000 7fe726d33640 1 --2- 172.21.15.47:0/166876171 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720095e60 0x7fe720132670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.192+0000 7fe726d33640 1 -- 172.21.15.47:0/166876171 >> 172.21.15.47:0/166876171 conn(0x7fe72009e050 msgr2=0x7fe72009e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.192+0000 7fe726d33640 1 -- 172.21.15.47:0/166876171 shutdown_connections 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.192+0000 7fe726d33640 1 -- 172.21.15.47:0/166876171 wait complete. 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.193+0000 7fe726d33640 1 Processor -- start 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.193+0000 7fe726d33640 1 -- start start 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.193+0000 7fe726d33640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe720095370 0x7fe7200717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.193+0000 7fe726d33640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720095e60 0x7fe720071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe726d33640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720132bb0 0x7fe720072240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe726d33640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe720135870 con 0x7fe720095370 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe726d33640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fe7201356f0 con 0x7fe720132bb0 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe726d33640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fe7201359f0 con 0x7fe720095e60 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe724acf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe720095370 0x7fe7200717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe71ffff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720095e60 0x7fe720071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe724acf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe720095370 0x7fe7200717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45814/0 (socket says 172.21.15.47:45814) 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe724acf640 1 -- 172.21.15.47:0/857007557 learned_addr learned my addr 172.21.15.47:0/857007557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe7252d0640 1 --2- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720132bb0 0x7fe720072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe71ffff640 1 -- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720132bb0 msgr2=0x7fe720072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe71ffff640 1 --2- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720132bb0 0x7fe720072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe71ffff640 1 -- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe720095370 msgr2=0x7fe7200717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe71ffff640 1 --2- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe720095370 0x7fe7200717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe71ffff640 1 -- 172.21.15.47:0/857007557 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe720094e80 con 0x7fe720095e60 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe724acf640 1 --2- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe720095370 0x7fe7200717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe71ffff640 1 --2- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720095e60 0x7fe720071d00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe70800e900 tx=0x7fe70800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe71d7fa640 1 -- 172.21.15.47:0/857007557 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708017070 con 0x7fe720095e60 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe71d7fa640 1 -- 172.21.15.47:0/857007557 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70800ca00 con 0x7fe720095e60 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe726d33640 1 -- 172.21.15.47:0/857007557 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72008de50 con 0x7fe720095e60 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe71d7fa640 1 -- 172.21.15.47:0/857007557 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70800cdf0 con 0x7fe720095e60 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.194+0000 7fe726d33640 1 -- 172.21.15.47:0/857007557 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fe720072780 con 0x7fe720095e60 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.195+0000 7fe726d33640 1 -- 172.21.15.47:0/857007557 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe720095750 con 0x7fe720095e60 2024-09-15T21:56:55.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.195+0000 7fe71d7fa640 1 -- 172.21.15.47:0/857007557 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fe70801a070 con 0x7fe720095e60 2024-09-15T21:56:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.196+0000 7fe71d7fa640 1 -- 172.21.15.47:0/857007557 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fe70805d7f0 con 0x7fe720095e60 2024-09-15T21:56:55.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.199+0000 7fe71d7fa640 1 -- 172.21.15.47:0/857007557 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fe720094e80 con 0x7fe720095e60 2024-09-15T21:56:55.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.347+0000 7fe726d33640 1 -- 172.21.15.47:0/857007557 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe72012fb00 con 0x7fe720095e60 2024-09-15T21:56:55.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.348+0000 7fe71d7fa640 1 -- 172.21.15.47:0/857007557 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+9172 (secure 0 0 0) 0x7fe70805c8a0 con 0x7fe720095e60 2024-09-15T21:56:55.350 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:55.350 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":31,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:56:54.740172+0000","last_up_change":"2024-09-15T21:56:48.715593+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":2079261393}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":2079261393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":2079261393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":2079261393}]},"public_addr":"172.21.15.47:6801/2079261393","cluster_addr":"172.21.15.47:6802/2079261393","heartbeat_back_addr":"172.21.15.47:6804/2079261393","heartbeat_front_addr":"172.21.15.47:6803/2079261393","state":["exists","up"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":25,"up_thru":25,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":4287109391}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":4287109391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":4287109391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":4287109391}]},"public_addr":"172.21.15.47:6805/4287109391","cluster_addr":"172.21.15.47:6806/4287109391","heartbeat_back_addr":"172.21.15.47:6808/4287109391","heartbeat_front_addr":"172.21.15.47:6807/4287109391","state":["exists","up"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":27,"up_thru":27,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":2903715449}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":2903715449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":2903715449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":2903715449}]},"public_addr":"172.21.15.47:6809/2903715449","cluster_addr":"172.21.15.47:6810/2903715449","heartbeat_back_addr":"172.21.15.47:6812/2903715449","heartbeat_front_addr":"172.21.15.47:6811/2903715449","state":["exists","up"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":27,"up_thru":27,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":2978958959}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":2978958959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":2978958959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":2978958959}]},"public_addr":"172.21.15.47:6813/2978958959","cluster_addr":"172.21.15.47:6814/2978958959","heartbeat_back_addr":"172.21.15.47:6816/2978958959","heartbeat_front_addr":"172.21.15.47:6815/2978958959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-15T21:56:40.690629+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/3200219561":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/2444109955":"2024-09-16T21:56:33.661886+0000","172.21.15.47:6800/32267":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1739125583":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1981014788":"2024-09-16T21:56:33.661886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:56:55.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.349+0000 7fe726d33640 1 -- 172.21.15.47:0/857007557 >> v1:172.21.15.47:6800/1790418198 conn(0x7fe700047fd0 legacy=0x7fe70004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.349+0000 7fe726d33640 1 -- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720095e60 msgr2=0x7fe720071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.349+0000 7fe726d33640 1 --2- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720095e60 0x7fe720071d00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe70800e900 tx=0x7fe70800edc0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.349+0000 7fe726d33640 1 -- 172.21.15.47:0/857007557 shutdown_connections 2024-09-15T21:56:55.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.349+0000 7fe726d33640 1 --2- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe720132bb0 0x7fe720072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.349+0000 7fe726d33640 1 --2- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe720095e60 0x7fe720071d00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.349+0000 7fe726d33640 1 --2- 172.21.15.47:0/857007557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe720095370 0x7fe7200717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.349+0000 7fe726d33640 1 -- 172.21.15.47:0/857007557 >> 172.21.15.47:0/857007557 conn(0x7fe72009e050 msgr2=0x7fe72012d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.349+0000 7fe726d33640 1 -- 172.21.15.47:0/857007557 shutdown_connections 2024-09-15T21:56:55.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.349+0000 7fe726d33640 1 -- 172.21.15.47:0/857007557 wait complete. 2024-09-15T21:56:55.363 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:56:55.363 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:56:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff54d9d4640 1 Processor -- start 2024-09-15T21:56:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff54d9d4640 1 -- start start 2024-09-15T21:56:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff54d9d4640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff548130440 0x7ff548134a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff54d9d4640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff548134fa0 0x7ff548137390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff54d9d4640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff54809c2c0 con 0x7ff54812f740 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff54d9d4640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff54806d080 con 0x7ff548130440 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff54d9d4640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7ff54806d200 con 0x7ff548134fa0 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff5477fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff548134fa0 0x7ff548137390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff5467fc640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff548130440 0x7ff548134a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff546ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff54812f740 0x7ff54812fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34328/0 (socket says 172.21.15.47:34328) 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.462+0000 7ff546ffd640 1 -- 172.21.15.47:0/196799664 learned_addr learned my addr 172.21.15.47:0/196799664 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.463+0000 7ff5477fe640 1 -- 172.21.15.47:0/196799664 >> v2:172.21.15.47:3300/0 conn(0x7ff548130440 msgr2=0x7ff548134a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.463+0000 7ff5477fe640 1 --2- 172.21.15.47:0/196799664 >> v2:172.21.15.47:3300/0 conn(0x7ff548130440 0x7ff548134a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.463+0000 7ff5477fe640 1 -- 172.21.15.47:0/196799664 >> v1:172.21.15.47:6789/0 conn(0x7ff54812f740 legacy=0x7ff54812fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.463+0000 7ff5477fe640 1 -- 172.21.15.47:0/196799664 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5481388e0 con 0x7ff548134fa0 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.463+0000 7ff545ffb640 1 -- 172.21.15.47:0/196799664 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1601814958 0 0) 0x7ff538001110 con 0x7ff54812f740 2024-09-15T21:56:55.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.463+0000 7ff5467fc640 1 --2- 172.21.15.47:0/196799664 >> v2:172.21.15.47:3300/0 conn(0x7ff548130440 0x7ff548134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.463+0000 7ff5477fe640 1 --2- 172.21.15.47:0/196799664 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff548134fa0 0x7ff548137390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff540009d90 tx=0x7ff54002a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c65454237b71d8b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff545ffb640 1 -- 172.21.15.47:0/196799664 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff540007740 con 0x7ff548134fa0 2024-09-15T21:56:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff545ffb640 1 -- 172.21.15.47:0/196799664 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5400078e0 con 0x7ff548134fa0 2024-09-15T21:56:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff545ffb640 1 -- 172.21.15.47:0/196799664 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff540007cb0 con 0x7ff548134fa0 2024-09-15T21:56:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff54d9d4640 1 -- 172.21.15.47:0/196799664 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff548134fa0 msgr2=0x7ff548137390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff54d9d4640 1 --2- 172.21.15.47:0/196799664 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff548134fa0 0x7ff548137390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff540009d90 tx=0x7ff54002a530 comp rx=0 tx=0).stop 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff54d9d4640 1 -- 172.21.15.47:0/196799664 shutdown_connections 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff54d9d4640 1 --2- 172.21.15.47:0/196799664 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff548134fa0 0x7ff548137390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff54d9d4640 1 --2- 172.21.15.47:0/196799664 >> v2:172.21.15.47:3300/0 conn(0x7ff548130440 0x7ff548134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff54d9d4640 1 -- 172.21.15.47:0/196799664 >> 172.21.15.47:0/196799664 conn(0x7ff54809edd0 msgr2=0x7ff54809f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff54d9d4640 1 -- 172.21.15.47:0/196799664 shutdown_connections 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.464+0000 7ff54d9d4640 1 -- 172.21.15.47:0/196799664 wait complete. 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.465+0000 7ff54d9d4640 1 Processor -- start 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.465+0000 7ff54d9d4640 1 -- start start 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff54d9d4640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff54812f740 0x7ff548098300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff54d9d4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff548134fa0 0x7ff548098840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff54d9d4640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff54812c7f0 0x7ff54812a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff54d9d4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff548137c60 con 0x7ff548134fa0 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff54d9d4640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7ff548137ae0 con 0x7ff54812f740 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff54d9d4640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7ff548137de0 con 0x7ff54812c7f0 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff5467fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff548134fa0 0x7ff548098840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff546ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff54812f740 0x7ff548098300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff5467fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff548134fa0 0x7ff548098840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45834/0 (socket says 172.21.15.47:45834) 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff5467fc640 1 -- 172.21.15.47:0/3654426811 learned_addr learned my addr 172.21.15.47:0/3654426811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff5477fe640 1 --2- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff54812c7f0 0x7ff54812a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff5467fc640 1 -- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff54812c7f0 msgr2=0x7ff54812a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff5467fc640 1 --2- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff54812c7f0 0x7ff54812a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff5467fc640 1 -- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff54812f740 msgr2=0x7ff548098300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff5467fc640 1 --2- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff54812f740 0x7ff548098300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff5467fc640 1 -- 172.21.15.47:0/3654426811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff548129fa0 con 0x7ff548134fa0 2024-09-15T21:56:55.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff5467fc640 1 --2- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff548134fa0 0x7ff548098840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00ebe0 tx=0x7ff53c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff52ffff640 1 -- 172.21.15.47:0/3654426811 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c017070 con 0x7ff548134fa0 2024-09-15T21:56:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff52ffff640 1 -- 172.21.15.47:0/3654426811 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53c00cc30 con 0x7ff548134fa0 2024-09-15T21:56:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff54d9d4640 1 -- 172.21.15.47:0/3654426811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5481312a0 con 0x7ff548134fa0 2024-09-15T21:56:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff52ffff640 1 -- 172.21.15.47:0/3654426811 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c013470 con 0x7ff548134fa0 2024-09-15T21:56:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.466+0000 7ff54d9d4640 1 -- 172.21.15.47:0/3654426811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54812ae00 con 0x7ff548134fa0 2024-09-15T21:56:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.467+0000 7ff54d9d4640 1 -- 172.21.15.47:0/3654426811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5481381f0 con 0x7ff548134fa0 2024-09-15T21:56:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.467+0000 7ff52ffff640 1 -- 172.21.15.47:0/3654426811 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7ff53c01a070 con 0x7ff548134fa0 2024-09-15T21:56:55.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.468+0000 7ff52ffff640 1 -- 172.21.15.47:0/3654426811 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7ff53c05d870 con 0x7ff548134fa0 2024-09-15T21:56:55.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.471+0000 7ff52ffff640 1 -- 172.21.15.47:0/3654426811 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7ff53c05c920 con 0x7ff548134fa0 2024-09-15T21:56:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.618+0000 7ff54d9d4640 1 -- 172.21.15.47:0/3654426811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff54812b8e0 con 0x7ff548134fa0 2024-09-15T21:56:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.619+0000 7ff52ffff640 1 -- 172.21.15.47:0/3654426811 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+9172 (secure 0 0 0) 0x7ff53c03ca60 con 0x7ff548134fa0 2024-09-15T21:56:55.621 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:55.622 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":31,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:56:54.740172+0000","last_up_change":"2024-09-15T21:56:48.715593+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":2079261393}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":2079261393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":2079261393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":2079261393}]},"public_addr":"172.21.15.47:6801/2079261393","cluster_addr":"172.21.15.47:6802/2079261393","heartbeat_back_addr":"172.21.15.47:6804/2079261393","heartbeat_front_addr":"172.21.15.47:6803/2079261393","state":["exists","up"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":25,"up_thru":25,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":4287109391}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":4287109391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":4287109391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":4287109391}]},"public_addr":"172.21.15.47:6805/4287109391","cluster_addr":"172.21.15.47:6806/4287109391","heartbeat_back_addr":"172.21.15.47:6808/4287109391","heartbeat_front_addr":"172.21.15.47:6807/4287109391","state":["exists","up"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":27,"up_thru":27,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":2903715449}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":2903715449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":2903715449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":2903715449}]},"public_addr":"172.21.15.47:6809/2903715449","cluster_addr":"172.21.15.47:6810/2903715449","heartbeat_back_addr":"172.21.15.47:6812/2903715449","heartbeat_front_addr":"172.21.15.47:6811/2903715449","state":["exists","up"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":27,"up_thru":27,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":2978958959}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":2978958959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":2978958959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":2978958959}]},"public_addr":"172.21.15.47:6813/2978958959","cluster_addr":"172.21.15.47:6814/2978958959","heartbeat_back_addr":"172.21.15.47:6816/2978958959","heartbeat_front_addr":"172.21.15.47:6815/2978958959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-15T21:56:40.690629+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/3200219561":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/2444109955":"2024-09-16T21:56:33.661886+0000","172.21.15.47:6800/32267":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1739125583":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1981014788":"2024-09-16T21:56:33.661886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:56:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.620+0000 7ff54d9d4640 1 -- 172.21.15.47:0/3654426811 >> v1:172.21.15.47:6800/1790418198 conn(0x7ff534048000 legacy=0x7ff53404a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.621+0000 7ff54d9d4640 1 -- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff548134fa0 msgr2=0x7ff548098840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.621+0000 7ff54d9d4640 1 --2- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff548134fa0 0x7ff548098840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00ebe0 tx=0x7ff53c00c520 comp rx=0 tx=0).stop 2024-09-15T21:56:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.621+0000 7ff54d9d4640 1 -- 172.21.15.47:0/3654426811 shutdown_connections 2024-09-15T21:56:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.621+0000 7ff54d9d4640 1 --2- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ff54812c7f0 0x7ff54812a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.621+0000 7ff54d9d4640 1 --2- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff548134fa0 0x7ff548098840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.621+0000 7ff54d9d4640 1 --2- 172.21.15.47:0/3654426811 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ff54812f740 0x7ff548098300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.621+0000 7ff54d9d4640 1 -- 172.21.15.47:0/3654426811 >> 172.21.15.47:0/3654426811 conn(0x7ff54809edd0 msgr2=0x7ff54809cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.621+0000 7ff54d9d4640 1 -- 172.21.15.47:0/3654426811 shutdown_connections 2024-09-15T21:56:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.621+0000 7ff54d9d4640 1 -- 172.21.15.47:0/3654426811 wait complete. 2024-09-15T21:56:55.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:56:55.634 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:56:55.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:56:55.635 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T21:56:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7f9c13c67640 1 Processor -- start 2024-09-15T21:56:55.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7fe30b6b7640 1 Processor -- start 2024-09-15T21:56:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7f92c5455640 1 Processor -- start 2024-09-15T21:56:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7fc25599c640 1 Processor -- start 2024-09-15T21:56:55.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7f92c5455640 1 -- start start 2024-09-15T21:56:55.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7fc25599c640 1 -- start start 2024-09-15T21:56:55.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7fe30b6b7640 1 -- start start 2024-09-15T21:56:55.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7f9c13c67640 1 -- start start 2024-09-15T21:56:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7f92c5455640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012ba10 0x7f92c012bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7f9c13c67640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7f9c13c67640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9c0c09af50 0x7f9c0c12f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.741+0000 7fc25599c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc25009d9a0 0x7fc25009dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f9c13c67640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9c0c12f920 0x7f9c0c12fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fe30b6b7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe30409a630 0x7fe30409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fe30b6b7640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30412faf0 0x7fe30412ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fe30b6b7640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe30409b600 con 0x7fe30409af50 2024-09-15T21:56:55.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f92c5455640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f92c0128c20 0x7f92c0133d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f92c5455640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92c0098230 con 0x7f92c012c330 2024-09-15T21:56:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc25599c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc25009e2c0 0x7fc250098090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f92c5455640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f92c0097e60 con 0x7f92c012ba10 2024-09-15T21:56:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f92c5455640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f92c0097680 con 0x7f92c0128c20 2024-09-15T21:56:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f92beffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012ba10 0x7f92c012bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f92be7fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f92c012c330 0x7f92c0128510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34334/0 (socket says 172.21.15.47:34334) 2024-09-15T21:56:55.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f92be7fc640 1 -- 172.21.15.47:0/3151505793 learned_addr learned my addr 172.21.15.47:0/3151505793 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:55.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fe30b6b7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe30406d480 con 0x7fe30409a630 2024-09-15T21:56:55.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fe30b6b7640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fe30406d140 con 0x7fe30412faf0 2024-09-15T21:56:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc25599c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2500985d0 0x7fc250098a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc25599c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc2500998f0 con 0x7fc25009e2c0 2024-09-15T21:56:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc24effd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc25009d9a0 0x7fc25009dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc24effd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc25009d9a0 0x7fc25009dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44710/0 (socket says 172.21.15.47:44710) 2024-09-15T21:56:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc25599c640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fc25006f6e0 con 0x7fc25009d9a0 2024-09-15T21:56:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc25599c640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fc25006fb00 con 0x7fc2500985d0 2024-09-15T21:56:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc24f7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2500985d0 0x7fc250098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc24effd640 1 -- 172.21.15.47:0/2107790443 learned_addr learned my addr 172.21.15.47:0/2107790443 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f9c13c67640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9c0c06c1f0 con 0x7f9c0c12f920 2024-09-15T21:56:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc24e7fc640 1 --2- 172.21.15.47:0/2107790443 >> v2:172.21.15.47:3300/0 conn(0x7fc25009e2c0 0x7fc250098090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f9c13c67640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c09b940 con 0x7f9c0c09a630 2024-09-15T21:56:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fe308c52640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe30409af50 0x7fe30412f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34350/0 (socket says 172.21.15.47:34350) 2024-09-15T21:56:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fe308c52640 1 -- 172.21.15.47:0/1218661553 learned_addr learned my addr 172.21.15.47:0/1218661553 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fe309c54640 1 --2- 172.21.15.47:0/1218661553 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30412faf0 0x7fe30412ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fe309453640 1 --2- 172.21.15.47:0/1218661553 >> v2:172.21.15.47:3300/0 conn(0x7fe30409a630 0x7fe30409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f9c13c67640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f9c0c130310 con 0x7f9c0c09af50 2024-09-15T21:56:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7fc24f7fe640 1 -- 172.21.15.47:0/2107790443 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc25009d9a0 msgr2=0x7fc25009dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7fc24f7fe640 1 --2- 172.21.15.47:0/2107790443 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc25009d9a0 0x7fc25009dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7fc24f7fe640 1 -- 172.21.15.47:0/2107790443 >> v2:172.21.15.47:3300/0 conn(0x7fc25009e2c0 msgr2=0x7fc250098090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7fc24f7fe640 1 --2- 172.21.15.47:0/2107790443 >> v2:172.21.15.47:3300/0 conn(0x7fc25009e2c0 0x7fc250098090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7fc24f7fe640 1 -- 172.21.15.47:0/2107790443 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc250099b20 con 0x7fc2500985d0 2024-09-15T21:56:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f9c11a03640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f9c11a03640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c09aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44718/0 (socket says 172.21.15.47:44718) 2024-09-15T21:56:55.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f9c11a03640 1 -- 172.21.15.47:0/1254773164 learned_addr learned my addr 172.21.15.47:0/1254773164 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f9c12204640 1 --2- 172.21.15.47:0/1254773164 >> v2:172.21.15.47:3300/0 conn(0x7f9c0c12f920 0x7f9c0c12fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7f9c11202640 1 --2- 172.21.15.47:0/1254773164 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9c0c09af50 0x7f9c0c12f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.742+0000 7f92bf7fe640 1 --2- 172.21.15.47:0/3151505793 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f92c0128c20 0x7f92c0133d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7f92beffd640 1 -- 172.21.15.47:0/3151505793 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f92c0128c20 msgr2=0x7f92c0133d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7f92beffd640 1 --2- 172.21.15.47:0/3151505793 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f92c0128c20 0x7f92c0133d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7f92beffd640 1 -- 172.21.15.47:0/3151505793 >> v1:172.21.15.47:6789/0 conn(0x7f92c012c330 legacy=0x7f92c0128510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7f92beffd640 1 -- 172.21.15.47:0/3151505793 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c0096d10 con 0x7f92c012ba10 2024-09-15T21:56:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7fe309453640 1 -- 172.21.15.47:0/1218661553 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30412faf0 msgr2=0x7fe30412ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7f9c11a03640 1 -- 172.21.15.47:0/1254773164 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9c0c09af50 msgr2=0x7f9c0c12f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7fe309453640 1 --2- 172.21.15.47:0/1218661553 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30412faf0 0x7fe30412ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7fe309453640 1 -- 172.21.15.47:0/1218661553 >> v1:172.21.15.47:6789/0 conn(0x7fe30409af50 legacy=0x7fe30412f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7fe309453640 1 -- 172.21.15.47:0/1218661553 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30409bc00 con 0x7fe30409a630 2024-09-15T21:56:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7f9c11a03640 1 --2- 172.21.15.47:0/1254773164 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9c0c09af50 0x7f9c0c12f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7f9c11a03640 1 -- 172.21.15.47:0/1254773164 >> v2:172.21.15.47:3300/0 conn(0x7f9c0c12f920 msgr2=0x7f9c0c12fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7f9c11a03640 1 --2- 172.21.15.47:0/1254773164 >> v2:172.21.15.47:3300/0 conn(0x7f9c0c12f920 0x7f9c0c12fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7f9c11a03640 1 -- 172.21.15.47:0/1254773164 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c09b390 con 0x7f9c0c09a630 2024-09-15T21:56:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.743+0000 7fc24f7fe640 1 --2- 172.21.15.47:0/2107790443 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2500985d0 0x7fc250098a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc248009d90 tx=0x7fc24802a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=803c30a5de4d0f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.744+0000 7fc24dffb640 1 -- 172.21.15.47:0/2107790443 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248007740 con 0x7fc2500985d0 2024-09-15T21:56:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.744+0000 7fc24dffb640 1 -- 172.21.15.47:0/2107790443 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2480078e0 con 0x7fc2500985d0 2024-09-15T21:56:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.744+0000 7fc24dffb640 1 -- 172.21.15.47:0/2107790443 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248007cb0 con 0x7fc2500985d0 2024-09-15T21:56:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.744+0000 7f92beffd640 1 --2- 172.21.15.47:0/3151505793 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012ba10 0x7f92c012bdf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f92b000a970 tx=0x7f92b002e700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2781a1bec16524a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.744+0000 7f9c11a03640 1 --2- 172.21.15.47:0/1254773164 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c09aa10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009870 tx=0x7f9c0802a090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e70e66c228c2b938 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.744+0000 7fe309453640 1 --2- 172.21.15.47:0/1218661553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30409a630 0x7fe30409aa10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe3000097a0 tx=0x7fe30002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1867cd862cdb75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.744+0000 7f92bdffb640 1 -- 172.21.15.47:0/3151505793 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b000b040 con 0x7f92c012ba10 2024-09-15T21:56:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.744+0000 7f92bdffb640 1 -- 172.21.15.47:0/3151505793 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b0033710 con 0x7f92c012ba10 2024-09-15T21:56:55.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.744+0000 7f92bdffb640 1 -- 172.21.15.47:0/3151505793 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b000b040 con 0x7f92c012ba10 2024-09-15T21:56:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.744+0000 7f9c10a01640 1 -- 172.21.15.47:0/1254773164 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08037070 con 0x7f9c0c09a630 2024-09-15T21:56:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f9c10a01640 1 -- 172.21.15.47:0/1254773164 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0802ade0 con 0x7f9c0c09a630 2024-09-15T21:56:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f9c10a01640 1 -- 172.21.15.47:0/1254773164 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08032660 con 0x7f9c0c09a630 2024-09-15T21:56:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f92c5455640 1 -- 172.21.15.47:0/3151505793 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012ba10 msgr2=0x7f92c012bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f92c5455640 1 --2- 172.21.15.47:0/3151505793 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012ba10 0x7f92c012bdf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f92b000a970 tx=0x7f92b002e700 comp rx=0 tx=0).stop 2024-09-15T21:56:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fe2fbfff640 1 -- 172.21.15.47:0/1218661553 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300037070 con 0x7fe30409a630 2024-09-15T21:56:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fe2fbfff640 1 -- 172.21.15.47:0/1218661553 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30002adb0 con 0x7fe30409a630 2024-09-15T21:56:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fe2fbfff640 1 -- 172.21.15.47:0/1218661553 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300032660 con 0x7fe30409a630 2024-09-15T21:56:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fc25599c640 1 -- 172.21.15.47:0/2107790443 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2500985d0 msgr2=0x7fc250098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fc25599c640 1 --2- 172.21.15.47:0/2107790443 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2500985d0 0x7fc250098a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc248009d90 tx=0x7fc24802a490 comp rx=0 tx=0).stop 2024-09-15T21:56:55.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fc25599c640 1 -- 172.21.15.47:0/2107790443 shutdown_connections 2024-09-15T21:56:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fc25599c640 1 --2- 172.21.15.47:0/2107790443 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc2500985d0 0x7fc250098a60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fc25599c640 1 --2- 172.21.15.47:0/2107790443 >> v2:172.21.15.47:3300/0 conn(0x7fc25009e2c0 0x7fc250098090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fc25599c640 1 --2- 172.21.15.47:0/2107790443 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc25009d9a0 0x7fc25009dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fc25599c640 1 -- 172.21.15.47:0/2107790443 >> 172.21.15.47:0/2107790443 conn(0x7fc2500a25c0 msgr2=0x7fc25006ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f92c5455640 1 -- 172.21.15.47:0/3151505793 shutdown_connections 2024-09-15T21:56:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f92c5455640 1 --2- 172.21.15.47:0/3151505793 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f92c0128c20 0x7f92c0133d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f92c5455640 1 --2- 172.21.15.47:0/3151505793 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012ba10 0x7f92c012bdf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f92c5455640 1 -- 172.21.15.47:0/3151505793 >> 172.21.15.47:0/3151505793 conn(0x7f92c009af50 msgr2=0x7f92c009b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fc25599c640 1 -- 172.21.15.47:0/2107790443 shutdown_connections 2024-09-15T21:56:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7fc25599c640 1 -- 172.21.15.47:0/2107790443 wait complete. 2024-09-15T21:56:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f92c5455640 1 -- 172.21.15.47:0/3151505793 shutdown_connections 2024-09-15T21:56:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f92c5455640 1 -- 172.21.15.47:0/3151505793 wait complete. 2024-09-15T21:56:55.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f9c13c67640 1 -- 172.21.15.47:0/1254773164 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 msgr2=0x7f9c0c09aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.745+0000 7f9c13c67640 1 --2- 172.21.15.47:0/1254773164 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c09aa10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009870 tx=0x7f9c0802a090 comp rx=0 tx=0).stop 2024-09-15T21:56:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7f9c13c67640 1 -- 172.21.15.47:0/1254773164 shutdown_connections 2024-09-15T21:56:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7fe30b6b7640 1 -- 172.21.15.47:0/1218661553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30409a630 msgr2=0x7fe30409aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7f9c13c67640 1 --2- 172.21.15.47:0/1254773164 >> v2:172.21.15.47:3300/0 conn(0x7f9c0c12f920 0x7f9c0c12fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7f9c13c67640 1 --2- 172.21.15.47:0/1254773164 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9c0c09af50 0x7f9c0c12f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7f9c13c67640 1 --2- 172.21.15.47:0/1254773164 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c09aa10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7f9c13c67640 1 -- 172.21.15.47:0/1254773164 >> 172.21.15.47:0/1254773164 conn(0x7f9c0c09e9b0 msgr2=0x7f9c0c09edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7fe30b6b7640 1 --2- 172.21.15.47:0/1218661553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30409a630 0x7fe30409aa10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe3000097a0 tx=0x7fe30002a060 comp rx=0 tx=0).stop 2024-09-15T21:56:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7fe30b6b7640 1 -- 172.21.15.47:0/1218661553 shutdown_connections 2024-09-15T21:56:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7fe30b6b7640 1 --2- 172.21.15.47:0/1218661553 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30412faf0 0x7fe30412ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7fe30b6b7640 1 --2- 172.21.15.47:0/1218661553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30409a630 0x7fe30409aa10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7fe30b6b7640 1 -- 172.21.15.47:0/1218661553 >> 172.21.15.47:0/1218661553 conn(0x7fe30409eb90 msgr2=0x7fe30409efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7f9c13c67640 1 -- 172.21.15.47:0/1254773164 shutdown_connections 2024-09-15T21:56:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7fe30b6b7640 1 -- 172.21.15.47:0/1218661553 shutdown_connections 2024-09-15T21:56:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7f9c13c67640 1 -- 172.21.15.47:0/1254773164 wait complete. 2024-09-15T21:56:55.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.747+0000 7fc25599c640 1 Processor -- start 2024-09-15T21:56:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.746+0000 7fe30b6b7640 1 -- 172.21.15.47:0/1218661553 wait complete. 2024-09-15T21:56:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.747+0000 7fc25599c640 1 -- start start 2024-09-15T21:56:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.747+0000 7f92c5455640 1 Processor -- start 2024-09-15T21:56:55.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.747+0000 7f92c5455640 1 -- start start 2024-09-15T21:56:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.747+0000 7fe30b6b7640 1 Processor -- start 2024-09-15T21:56:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.747+0000 7fe30b6b7640 1 -- start start 2024-09-15T21:56:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc25599c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc2500985d0 0x7fc2501272c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc25599c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc25009d9a0 0x7fc250127800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f92c5455640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0128c20 0x7f92c0070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fe30b6b7640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30409a630 0x7fe3041c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f92c5455640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f92c012ba10 0x7f92c0070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f9c13c67640 1 Processor -- start 2024-09-15T21:56:55.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc25599c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc25009e2c0 0x7fc250127d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f92c5455640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012c330 0x7f92c0071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f9c13c67640 1 -- start start 2024-09-15T21:56:55.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fe30b6b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30409af50 0x7fe3041c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc24effd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc2500985d0 0x7fc2501272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc24effd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc2500985d0 0x7fc2501272c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44732/0 (socket says 172.21.15.47:44732) 2024-09-15T21:56:55.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fe309453640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30409a630 0x7fe3041c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fe309453640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30409a630 0x7fe3041c4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:42080/0 (socket says 172.21.15.47:42080) 2024-09-15T21:56:55.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc24e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc25009d9a0 0x7fc250127800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f92c5455640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f92c0129230 con 0x7f92c0128c20 2024-09-15T21:56:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f92c5455640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f92c01290b0 con 0x7f92c012c330 2024-09-15T21:56:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f92c5455640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f92c0099ea0 con 0x7f92c012ba10 2024-09-15T21:56:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f92beffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0128c20 0x7f92c0070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fe308c52640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30409af50 0x7fe3041c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc24e7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc25009d9a0 0x7fc250127800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45880/0 (socket says 172.21.15.47:45880) 2024-09-15T21:56:55.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc24f7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc25009e2c0 0x7fc250127d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f92beffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0128c20 0x7f92c0070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45872/0 (socket says 172.21.15.47:45872) 2024-09-15T21:56:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f92be7fc640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f92c012ba10 0x7f92c0070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7f92beffd640 1 -- 172.21.15.47:0/1161368136 learned_addr learned my addr 172.21.15.47:0/1161368136 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc24f7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc25009e2c0 0x7fc250127d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:42088/0 (socket says 172.21.15.47:42088) 2024-09-15T21:56:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc25599c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc25009eca0 con 0x7fc25009d9a0 2024-09-15T21:56:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc25599c640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fc250136a00 con 0x7fc2500985d0 2024-09-15T21:56:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc25599c640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fc250129b60 con 0x7fc25009e2c0 2024-09-15T21:56:55.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fc24effd640 1 -- 172.21.15.47:0/1356161001 learned_addr learned my addr 172.21.15.47:0/1356161001 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fe30b6b7640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe30412faf0 0x7fe3041311d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fe30b6b7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe304137ad0 con 0x7fe30409af50 2024-09-15T21:56:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fe30b6b7640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fe304137950 con 0x7fe30412faf0 2024-09-15T21:56:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fe30b6b7640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fe304137c50 con 0x7fe30409a630 2024-09-15T21:56:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.748+0000 7fe309453640 1 -- 172.21.15.47:0/2339911107 learned_addr learned my addr 172.21.15.47:0/2339911107 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f9c13c67640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c127960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fe309453640 1 -- 172.21.15.47:0/2339911107 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe30412faf0 msgr2=0x7fe3041311d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fe309453640 1 --2- 172.21.15.47:0/2339911107 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe30412faf0 0x7fe3041311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fe309453640 1 -- 172.21.15.47:0/2339911107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30409af50 msgr2=0x7fe3041c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fe309453640 1 --2- 172.21.15.47:0/2339911107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30409af50 0x7fe3041c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fe309453640 1 -- 172.21.15.47:0/2339911107 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe304132cc0 con 0x7fe30409a630 2024-09-15T21:56:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fc24e7fc640 1 -- 172.21.15.47:0/1356161001 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc25009e2c0 msgr2=0x7fc250127d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fc24e7fc640 1 --2- 172.21.15.47:0/1356161001 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc25009e2c0 0x7fc250127d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fc24e7fc640 1 -- 172.21.15.47:0/1356161001 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc2500985d0 msgr2=0x7fc2501272c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fc24e7fc640 1 --2- 172.21.15.47:0/1356161001 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc2500985d0 0x7fc2501272c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fc24e7fc640 1 -- 172.21.15.47:0/1356161001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc250129880 con 0x7fc25009d9a0 2024-09-15T21:56:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f92be7fc640 1 -- 172.21.15.47:0/1161368136 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012c330 msgr2=0x7f92c0071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f92bf7fe640 1 --2- 172.21.15.47:0/1161368136 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012c330 0x7f92c0071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f9c11a03640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c127960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f92be7fc640 1 --2- 172.21.15.47:0/1161368136 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012c330 0x7f92c0071310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f9c11a03640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c127960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44760/0 (socket says 172.21.15.47:44760) 2024-09-15T21:56:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f9c11a03640 1 -- 172.21.15.47:0/713542975 learned_addr learned my addr 172.21.15.47:0/713542975 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f9c13c67640 1 --2- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0c09af50 0x7f9c0c127ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fe309453640 1 --2- 172.21.15.47:0/2339911107 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30409a630 0x7fe3041c4d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe300035040 tx=0x7fe3000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f92be7fc640 1 -- 172.21.15.47:0/1161368136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0128c20 msgr2=0x7f92c0070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f92be7fc640 1 --2- 172.21.15.47:0/1161368136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0128c20 0x7f92c0070890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f92be7fc640 1 -- 172.21.15.47:0/1161368136 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c0072a30 con 0x7f92c012ba10 2024-09-15T21:56:55.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7fc24e7fc640 1 --2- 172.21.15.47:0/1356161001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc25009d9a0 0x7fc250127800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc24400d5f0 tx=0x7fc24400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.749+0000 7f9c13c67640 1 --2- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9c0c12f920 0x7f9c0c1283e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c136710 con 0x7f9c0c09af50 2024-09-15T21:56:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c09c140 con 0x7f9c0c09a630 2024-09-15T21:56:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f9c0c136890 con 0x7f9c0c12f920 2024-09-15T21:56:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c12204640 1 --2- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9c0c12f920 0x7f9c0c1283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c11202640 1 --2- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0c09af50 0x7f9c0c127ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f92be7fc640 1 --2- 172.21.15.47:0/1161368136 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f92c012ba10 0x7f92c0070dd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f92b400c440 tx=0x7f92b4001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7fe2f9ffb640 1 -- 172.21.15.47:0/2339911107 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300041020 con 0x7fe30409a630 2024-09-15T21:56:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7fe2f9ffb640 1 -- 172.21.15.47:0/2339911107 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe30003c670 con 0x7fe30409a630 2024-09-15T21:56:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3041c57c0 con 0x7fe30409a630 2024-09-15T21:56:55.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7fe2f9ffb640 1 -- 172.21.15.47:0/2339911107 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300041020 con 0x7fe30409a630 2024-09-15T21:56:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f92beffd640 1 --2- 172.21.15.47:0/1161368136 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0128c20 0x7f92c0070890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:55.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f92bf7fe640 1 --2- 172.21.15.47:0/1161368136 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f92c012c330 0x7f92c0071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:56:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c12204640 1 -- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 msgr2=0x7f9c0c127960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c12204640 1 --2- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c127960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c12204640 1 -- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0c09af50 msgr2=0x7f9c0c127ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c12204640 1 --2- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c0c09af50 0x7f9c0c127ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c12204640 1 -- 172.21.15.47:0/713542975 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c129c20 con 0x7f9c0c12f920 2024-09-15T21:56:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c11a03640 1 --2- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9c0c09a630 0x7f9c0c127960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:55.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fe304133f80 con 0x7fe30409a630 2024-09-15T21:56:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.750+0000 7f9c12204640 1 --2- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9c0c12f920 0x7f9c0c1283e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9c04007d40 tx=0x7f9c0400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.751+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe30406fbb0 con 0x7fe30409a630 2024-09-15T21:56:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7fc237fff640 1 -- 172.21.15.47:0/1356161001 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440125f0 con 0x7fc25009d9a0 2024-09-15T21:56:55.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7f9c027fc640 1 -- 172.21.15.47:0/713542975 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0400ed70 con 0x7f9c0c12f920 2024-09-15T21:56:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7fc237fff640 1 -- 172.21.15.47:0/1356161001 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc244019070 con 0x7fc25009d9a0 2024-09-15T21:56:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c12cd80 con 0x7f9c0c12f920 2024-09-15T21:56:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7f92a77fe640 1 -- 172.21.15.47:0/1161368136 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b4017070 con 0x7f92c012ba10 2024-09-15T21:56:55.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc250129410 con 0x7fc25009d9a0 2024-09-15T21:56:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c0126200 con 0x7f92c012ba10 2024-09-15T21:56:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c12b2d0 con 0x7f9c0c12f920 2024-09-15T21:56:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c0073270 con 0x7f92c012ba10 2024-09-15T21:56:55.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7fc237fff640 1 -- 172.21.15.47:0/1356161001 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440129d0 con 0x7fc25009d9a0 2024-09-15T21:56:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7f9c027fc640 1 -- 172.21.15.47:0/713542975 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c04003990 con 0x7f9c0c12f920 2024-09-15T21:56:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.753+0000 7f9c027fc640 1 -- 172.21.15.47:0/713542975 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04003d40 con 0x7f9c0c12f920 2024-09-15T21:56:55.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.754+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc250128ff0 con 0x7fc25009d9a0 2024-09-15T21:56:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.754+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f92c00691a0 con 0x7f92c012ba10 2024-09-15T21:56:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.754+0000 7f92a77fe640 1 -- 172.21.15.47:0/1161368136 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b4005ce0 con 0x7f92c012ba10 2024-09-15T21:56:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.754+0000 7f92a77fe640 1 -- 172.21.15.47:0/1161368136 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b40072d0 con 0x7f92c012ba10 2024-09-15T21:56:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.755+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c0c12f3e0 con 0x7f9c0c12f920 2024-09-15T21:56:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.754+0000 7fe2f9ffb640 1 -- 172.21.15.47:0/2339911107 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fe300044020 con 0x7fe30409a630 2024-09-15T21:56:55.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.755+0000 7fc237fff640 1 -- 172.21.15.47:0/1356161001 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fc244012b70 con 0x7fc25009d9a0 2024-09-15T21:56:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.755+0000 7fe2f9ffb640 1 -- 172.21.15.47:0/2339911107 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fe30004f7c0 con 0x7fe30409a630 2024-09-15T21:56:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.755+0000 7f92a77fe640 1 -- 172.21.15.47:0/1161368136 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f92b401a070 con 0x7f92c012ba10 2024-09-15T21:56:55.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.755+0000 7f9c027fc640 1 -- 172.21.15.47:0/713542975 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f9c0401b020 con 0x7f9c0c12f920 2024-09-15T21:56:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.755+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc25009dd80 con 0x7fc25009d9a0 2024-09-15T21:56:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.755+0000 7fe2f9ffb640 1 -- 172.21.15.47:0/2339911107 --> v1:172.21.15.47:6809/2903715449 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2e8052d40 con 0x7fe2e804f030 2024-09-15T21:56:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.755+0000 7fe2f9ffb640 1 -- 172.21.15.47:0/2339911107 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe300046d60 con 0x7fe30409a630 2024-09-15T21:56:55.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.756+0000 7f92a77fe640 1 -- 172.21.15.47:0/1161368136 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f92b405cd70 con 0x7f92c012ba10 2024-09-15T21:56:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.756+0000 7fc237fff640 1 -- 172.21.15.47:0/1356161001 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fc244062db0 con 0x7fc25009d9a0 2024-09-15T21:56:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.756+0000 7f92a77fe640 1 -- 172.21.15.47:0/1161368136 --> v1:172.21.15.47:6805/4287109391 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92ac052500 con 0x7f92ac04e850 2024-09-15T21:56:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.756+0000 7f92a77fe640 1 -- 172.21.15.47:0/1161368136 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f92b405f390 con 0x7f92c012ba10 2024-09-15T21:56:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.756+0000 7fc237fff640 1 -- 172.21.15.47:0/1356161001 --> v1:172.21.15.47:6813/2978958959 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc24400af30 con 0x7fc23c04e940 2024-09-15T21:56:55.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.756+0000 7fc237fff640 1 -- 172.21.15.47:0/1356161001 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc244063060 con 0x7fc25009d9a0 2024-09-15T21:56:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.756+0000 7f9c027fc640 1 -- 172.21.15.47:0/713542975 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f9c0405da60 con 0x7f9c0c12f920 2024-09-15T21:56:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.756+0000 7f9c027fc640 1 -- 172.21.15.47:0/713542975 --> v1:172.21.15.47:6801/2079261393 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c04016e50 con 0x7f9bf004e8f0 2024-09-15T21:56:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.756+0000 7f9c027fc640 1 -- 172.21.15.47:0/713542975 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9c040622a0 con 0x7f9c0c12f920 2024-09-15T21:56:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.757+0000 7fe2f9ffb640 1 -- 172.21.15.47:0/2339911107 <== osd.2 v1:172.21.15.47:6809/2903715449 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fe2e8052d40 con 0x7fe2e804f030 2024-09-15T21:56:55.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.757+0000 7f9c027fc640 1 -- 172.21.15.47:0/713542975 <== osd.0 v1:172.21.15.47:6801/2079261393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f9c04016e50 con 0x7f9bf004e8f0 2024-09-15T21:56:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.757+0000 7fc237fff640 1 -- 172.21.15.47:0/1356161001 <== osd.3 v1:172.21.15.47:6813/2978958959 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fc24400af30 con 0x7fc23c04e940 2024-09-15T21:56:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.757+0000 7f92a77fe640 1 -- 172.21.15.47:0/1161368136 <== osd.1 v1:172.21.15.47:6805/4287109391 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f92ac052500 con 0x7f92ac04e850 2024-09-15T21:56:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.771+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 --> v1:172.21.15.47:6813/2978958959 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc25012fc30 con 0x7fc23c04e940 2024-09-15T21:56:55.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.771+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 --> v1:172.21.15.47:6801/2079261393 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c0c12b720 con 0x7f9bf004e8f0 2024-09-15T21:56:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.771+0000 7fc237fff640 1 -- 172.21.15.47:0/1356161001 <== osd.3 v1:172.21.15.47:6813/2978958959 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 3324758384) 0x7fc25012fc30 con 0x7fc23c04e940 2024-09-15T21:56:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.771+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 --> v1:172.21.15.47:6805/4287109391 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92c0062880 con 0x7f92ac04e850 2024-09-15T21:56:55.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.771+0000 7f9c027fc640 1 -- 172.21.15.47:0/713542975 <== osd.0 v1:172.21.15.47:6801/2079261393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 662925176) 0x7f9c0c12b720 con 0x7f9bf004e8f0 2024-09-15T21:56:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.771+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 >> v1:172.21.15.47:6813/2978958959 conn(0x7fc23c04e940 legacy=0x7fc23c050d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.771+0000 7f92a77fe640 1 -- 172.21.15.47:0/1161368136 <== osd.1 v1:172.21.15.47:6805/4287109391 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 662925176) 0x7f92c0062880 con 0x7f92ac04e850 2024-09-15T21:56:55.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.771+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 --> v1:172.21.15.47:6809/2903715449 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe304131990 con 0x7fe2e804f030 2024-09-15T21:56:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.771+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 >> v1:172.21.15.47:6800/1790418198 conn(0x7fc23c048050 legacy=0x7fc23c04a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc25009d9a0 msgr2=0x7fc250127800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 >> v1:172.21.15.47:6801/2079261393 conn(0x7f9bf004e8f0 legacy=0x7f9bf0050d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fc25599c640 1 --2- 172.21.15.47:0/1356161001 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc25009d9a0 0x7fc250127800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc24400d5f0 tx=0x7fc24400dab0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fc24f7fe640 1 -- 172.21.15.47:0/1356161001 reap_dead start 2024-09-15T21:56:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 >> v1:172.21.15.47:6800/1790418198 conn(0x7f9bf0048000 legacy=0x7f9bf004a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9c0c12f920 msgr2=0x7f9c0c1283e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f9c13c67640 1 --2- 172.21.15.47:0/713542975 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9c0c12f920 0x7f9c0c1283e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9c04007d40 tx=0x7f9c0400e4b0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fe2f9ffb640 1 -- 172.21.15.47:0/2339911107 <== osd.2 v1:172.21.15.47:6809/2903715449 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 3324758384) 0x7fe304131990 con 0x7fe2e804f030 2024-09-15T21:56:55.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 shutdown_connections 2024-09-15T21:56:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 >> v1:172.21.15.47:6805/4287109391 conn(0x7f92ac04e850 legacy=0x7f92ac050c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 >> 172.21.15.47:0/1356161001 conn(0x7fc2500a25c0 msgr2=0x7fc25012f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f9c12204640 1 -- 172.21.15.47:0/713542975 reap_dead start 2024-09-15T21:56:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 shutdown_connections 2024-09-15T21:56:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 >> 172.21.15.47:0/713542975 conn(0x7f9c0c09e9b0 msgr2=0x7f9c0c06edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 shutdown_connections 2024-09-15T21:56:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 shutdown_connections 2024-09-15T21:56:55.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 >> v1:172.21.15.47:6800/1790418198 conn(0x7f92ac047fd0 legacy=0x7f92ac04a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f92c012ba10 msgr2=0x7f92c0070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f92c5455640 1 --2- 172.21.15.47:0/1161368136 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f92c012ba10 0x7f92c0070dd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f92b400c440 tx=0x7f92b4001e70 comp rx=0 tx=0).stop 2024-09-15T21:56:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f92bf7fe640 1 -- 172.21.15.47:0/1161368136 reap_dead start 2024-09-15T21:56:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fc25599c640 1 -- 172.21.15.47:0/1356161001 wait complete. 2024-09-15T21:56:55.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f9c13c67640 1 -- 172.21.15.47:0/713542975 wait complete. 2024-09-15T21:56:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 shutdown_connections 2024-09-15T21:56:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 >> 172.21.15.47:0/1161368136 conn(0x7f92c009af50 msgr2=0x7f92c012b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 >> v1:172.21.15.47:6809/2903715449 conn(0x7fe2e804f030 legacy=0x7fe2e80514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 shutdown_connections 2024-09-15T21:56:55.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7f92c5455640 1 -- 172.21.15.47:0/1161368136 wait complete. 2024-09-15T21:56:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 >> v1:172.21.15.47:6800/1790418198 conn(0x7fe2e80481d0 legacy=0x7fe2e804a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30409a630 msgr2=0x7fe3041c4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.772+0000 7fe30b6b7640 1 --2- 172.21.15.47:0/2339911107 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe30409a630 0x7fe3041c4d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe300035040 tx=0x7fe3000046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.773+0000 7fe309c54640 1 -- 172.21.15.47:0/2339911107 reap_dead start 2024-09-15T21:56:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.773+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 shutdown_connections 2024-09-15T21:56:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.773+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 >> 172.21.15.47:0/2339911107 conn(0x7fe30409eb90 msgr2=0x7fe30412f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.773+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 shutdown_connections 2024-09-15T21:56:55.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.773+0000 7fe30b6b7640 1 -- 172.21.15.47:0/2339911107 wait complete. 2024-09-15T21:56:55.802 INFO:teuthology.orchestra.run.smithi047.stdout:107374182404 2024-09-15T21:56:55.802 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:56:55.803 INFO:teuthology.orchestra.run.smithi047.stdout:115964116995 2024-09-15T21:56:55.803 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:56:55.804 INFO:teuthology.orchestra.run.smithi047.stdout:107374182404 2024-09-15T21:56:55.804 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:56:55.805 INFO:teuthology.orchestra.run.smithi047.stdout:115964116995 2024-09-15T21:56:55.806 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:56:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.947+0000 7f7b139a3640 1 Processor -- start 2024-09-15T21:56:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.947+0000 7f7e97a0d640 1 Processor -- start 2024-09-15T21:56:55.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.947+0000 7f9f06b2a640 1 Processor -- start 2024-09-15T21:56:55.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.947+0000 7f7b139a3640 1 -- start start 2024-09-15T21:56:55.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.947+0000 7faa0cbfd640 1 Processor -- start 2024-09-15T21:56:55.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.947+0000 7f7e97a0d640 1 -- start start 2024-09-15T21:56:55.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.947+0000 7f9f06b2a640 1 -- start start 2024-09-15T21:56:55.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7faa0cbfd640 1 -- start start 2024-09-15T21:56:55.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b139a3640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b0c09a630 0x7f7b0c09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b139a3640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09af50 0x7f7b0c12f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b139a3640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c12f920 0x7f7b0c12fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b139a3640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7b0c09b350 con 0x7f7b0c09a630 2024-09-15T21:56:55.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b139a3640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0c09b600 con 0x7f7b0c12f920 2024-09-15T21:56:55.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b139a3640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f7b0c06d480 con 0x7f7b0c09af50 2024-09-15T21:56:55.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7e97a0d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90096b00 0x7f7e90096ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7e97a0d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e90097420 0x7f7e90094050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7e97a0d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7e90094590 0x7f7e90094a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f9f06b2a640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0012fbf0 0x7f9f0012ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b1173f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b0c09a630 0x7f7b0c09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b1173f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b0c09a630 0x7f7b0c09aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45894/0 (socket says 172.21.15.47:45894) 2024-09-15T21:56:55.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b1173f640 1 -- 172.21.15.47:0/1087343832 learned_addr learned my addr 172.21.15.47:0/1087343832 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7faa0cbfd640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7faa08097500 0x7faa080978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b10f3e640 1 --2- 172.21.15.47:0/1087343832 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09af50 0x7f7b0c12f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7faa0cbfd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08098220 0x7faa08137650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f9f06b2a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9f0006e890 0x7f9f00137800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f9f06b2a640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f0012f730 con 0x7f9f0006dd00 2024-09-15T21:56:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f9f06b2a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9f00075b40 con 0x7f9f0006e890 2024-09-15T21:56:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f9f06b2a640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f9f00099b90 con 0x7f9f0012fbf0 2024-09-15T21:56:55.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f9f048c6640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0012fbf0 0x7f9f0012ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7b11f40640 1 --2- 172.21.15.47:0/1087343832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c12f920 0x7f7b0c12fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7e97a0d640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7e9006e1a0 con 0x7f7e90094590 2024-09-15T21:56:55.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7faa0cbfd640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa08091460 con 0x7faa08097e20 2024-09-15T21:56:55.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7e97a0d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9006cfd0 con 0x7f7e90097420 2024-09-15T21:56:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f7e97a0d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f7e9006ccf0 con 0x7f7e90096b00 2024-09-15T21:56:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7faa0cbfd640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7faa08098950 con 0x7faa08097500 2024-09-15T21:56:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7faa0cbfd640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7faa08099290 con 0x7faa08098220 2024-09-15T21:56:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f9f048c6640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0012fbf0 0x7f9f0012ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44804/0 (socket says 172.21.15.47:44804) 2024-09-15T21:56:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.948+0000 7f9f048c6640 1 -- 172.21.15.47:0/3386299727 learned_addr learned my addr 172.21.15.47:0/3386299727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7faa06d9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08098220 0x7faa08137650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e957a9640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90096b00 0x7f7e90096ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7faa06d9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08098220 0x7faa08137650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44812/0 (socket says 172.21.15.47:44812) 2024-09-15T21:56:55.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e957a9640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90096b00 0x7f7e90096ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:42098/0 (socket says 172.21.15.47:42098) 2024-09-15T21:56:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e957a9640 1 -- 172.21.15.47:0/2006866445 learned_addr learned my addr 172.21.15.47:0/2006866445 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7b10f3e640 1 -- 172.21.15.47:0/1087343832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c12f920 msgr2=0x7f7b0c12fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7b10f3e640 1 --2- 172.21.15.47:0/1087343832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c12f920 0x7f7b0c12fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e94fa8640 1 --2- 172.21.15.47:0/2006866445 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e90097420 0x7f7e90094050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7faa06d9d640 1 -- 172.21.15.47:0/264741966 learned_addr learned my addr 172.21.15.47:0/264741966 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7b10f3e640 1 -- 172.21.15.47:0/1087343832 >> v2:172.21.15.47:3300/0 conn(0x7f7b0c09a630 msgr2=0x7f7b0c09aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7b10f3e640 1 --2- 172.21.15.47:0/1087343832 >> v2:172.21.15.47:3300/0 conn(0x7f7b0c09a630 0x7f7b0c09aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f9f048c6640 1 -- 172.21.15.47:0/3386299727 >> v2:172.21.15.47:3300/0 conn(0x7f9f0006e890 msgr2=0x7f9f00137800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f9f048c6640 1 --2- 172.21.15.47:0/3386299727 >> v2:172.21.15.47:3300/0 conn(0x7f9f0006e890 0x7f9f00137800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f9f048c6640 1 -- 172.21.15.47:0/3386299727 >> v1:172.21.15.47:6789/0 conn(0x7f9f0006dd00 legacy=0x7f9f0006e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7b10f3e640 1 -- 172.21.15.47:0/1087343832 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c06d140 con 0x7f7b0c09af50 2024-09-15T21:56:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f9f048c6640 1 -- 172.21.15.47:0/3386299727 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0006ee90 con 0x7f9f0012fbf0 2024-09-15T21:56:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7faa06d9d640 1 -- 172.21.15.47:0/264741966 >> v2:172.21.15.47:3300/0 conn(0x7faa08097500 msgr2=0x7faa080978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7faa06d9d640 1 --2- 172.21.15.47:0/264741966 >> v2:172.21.15.47:3300/0 conn(0x7faa08097500 0x7faa080978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7faa06d9d640 1 -- 172.21.15.47:0/264741966 >> v1:172.21.15.47:6789/0 conn(0x7faa08097e20 legacy=0x7faa08133f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e95faa640 1 --2- 172.21.15.47:0/2006866445 >> v2:172.21.15.47:3300/0 conn(0x7f7e90094590 0x7f7e90094a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e94fa8640 1 -- 172.21.15.47:0/2006866445 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90096b00 msgr2=0x7f7e90096ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7faa06d9d640 1 -- 172.21.15.47:0/264741966 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa08138ba0 con 0x7faa08098220 2024-09-15T21:56:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e94fa8640 1 --2- 172.21.15.47:0/2006866445 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90096b00 0x7f7e90096ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e94fa8640 1 -- 172.21.15.47:0/2006866445 >> v2:172.21.15.47:3300/0 conn(0x7f7e90094590 msgr2=0x7f7e90094a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:55.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e94fa8640 1 --2- 172.21.15.47:0/2006866445 >> v2:172.21.15.47:3300/0 conn(0x7f7e90094590 0x7f7e90094a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e94fa8640 1 -- 172.21.15.47:0/2006866445 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9006de90 con 0x7f7e90097420 2024-09-15T21:56:55.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e957a9640 1 --2- 172.21.15.47:0/2006866445 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90096b00 0x7f7e90096ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:55.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.949+0000 7f7e95faa640 1 --2- 172.21.15.47:0/2006866445 >> v2:172.21.15.47:3300/0 conn(0x7f7e90094590 0x7f7e90094a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:56:55.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7faa06d9d640 1 --2- 172.21.15.47:0/264741966 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08098220 0x7faa08137650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faa00009e50 tx=0x7faa0002a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd75b70920b3e6d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7f9f048c6640 1 --2- 172.21.15.47:0/3386299727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0012fbf0 0x7f9f0012ffd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009870 tx=0x7f9ef402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=690bae1d577adf60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7f7b10f3e640 1 --2- 172.21.15.47:0/1087343832 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09af50 0x7f7b0c12f3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0097a0 tx=0x7f7afc029fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=371a20adcc47381e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7f7e94fa8640 1 --2- 172.21.15.47:0/2006866445 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e90097420 0x7f7e90094050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e800097d0 tx=0x7f7e8002a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d85d8eca6b26bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7faa0559a640 1 -- 172.21.15.47:0/264741966 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00036070 con 0x7faa08098220 2024-09-15T21:56:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7faa0559a640 1 -- 172.21.15.47:0/264741966 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa00031600 con 0x7faa08098220 2024-09-15T21:56:55.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7faa0559a640 1 -- 172.21.15.47:0/264741966 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000319d0 con 0x7faa08098220 2024-09-15T21:56:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7f7b03fff640 1 -- 172.21.15.47:0/1087343832 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc037070 con 0x7f7b0c09af50 2024-09-15T21:56:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7f9eff7fe640 1 -- 172.21.15.47:0/3386299727 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4037070 con 0x7f9f0012fbf0 2024-09-15T21:56:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7f7b03fff640 1 -- 172.21.15.47:0/1087343832 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7afc02ad20 con 0x7f7b0c09af50 2024-09-15T21:56:55.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7f9eff7fe640 1 -- 172.21.15.47:0/3386299727 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef4032440 con 0x7f9f0012fbf0 2024-09-15T21:56:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.950+0000 7f7b03fff640 1 -- 172.21.15.47:0/1087343832 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc037070 con 0x7f7b0c09af50 2024-09-15T21:56:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f9eff7fe640 1 -- 172.21.15.47:0/3386299727 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4037070 con 0x7f9f0012fbf0 2024-09-15T21:56:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f9f06b2a640 1 -- 172.21.15.47:0/3386299727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0012fbf0 msgr2=0x7f9f0012ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f9f06b2a640 1 --2- 172.21.15.47:0/3386299727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0012fbf0 0x7f9f0012ffd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009870 tx=0x7f9ef402a180 comp rx=0 tx=0).stop 2024-09-15T21:56:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f7e87fff640 1 -- 172.21.15.47:0/2006866445 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80037070 con 0x7f7e90097420 2024-09-15T21:56:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f7e87fff640 1 -- 172.21.15.47:0/2006866445 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8002ad10 con 0x7f7e90097420 2024-09-15T21:56:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7faa0cbfd640 1 -- 172.21.15.47:0/264741966 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08098220 msgr2=0x7faa08137650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7faa0cbfd640 1 --2- 172.21.15.47:0/264741966 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08098220 0x7faa08137650 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faa00009e50 tx=0x7faa0002a610 comp rx=0 tx=0).stop 2024-09-15T21:56:55.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f7e87fff640 1 -- 172.21.15.47:0/2006866445 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80032660 con 0x7f7e90097420 2024-09-15T21:56:55.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f9f06b2a640 1 -- 172.21.15.47:0/3386299727 shutdown_connections 2024-09-15T21:56:55.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f9f06b2a640 1 --2- 172.21.15.47:0/3386299727 >> v2:172.21.15.47:3300/0 conn(0x7f9f0006e890 0x7f9f00137800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f9f06b2a640 1 --2- 172.21.15.47:0/3386299727 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0012fbf0 0x7f9f0012ffd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f9f06b2a640 1 -- 172.21.15.47:0/3386299727 >> 172.21.15.47:0/3386299727 conn(0x7f9f0009ece0 msgr2=0x7f9f0009f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f9f06b2a640 1 -- 172.21.15.47:0/3386299727 shutdown_connections 2024-09-15T21:56:55.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f7b139a3640 1 -- 172.21.15.47:0/1087343832 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09af50 msgr2=0x7f7b0c12f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f7b139a3640 1 --2- 172.21.15.47:0/1087343832 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09af50 0x7f7b0c12f3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0097a0 tx=0x7f7afc029fd0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f9f06b2a640 1 -- 172.21.15.47:0/3386299727 wait complete. 2024-09-15T21:56:55.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.951+0000 7f7b139a3640 1 -- 172.21.15.47:0/1087343832 shutdown_connections 2024-09-15T21:56:55.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7b139a3640 1 --2- 172.21.15.47:0/1087343832 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c12f920 0x7f7b0c12fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7b139a3640 1 --2- 172.21.15.47:0/1087343832 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09af50 0x7f7b0c12f3e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7b139a3640 1 --2- 172.21.15.47:0/1087343832 >> v2:172.21.15.47:3300/0 conn(0x7f7b0c09a630 0x7f7b0c09aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7faa0cbfd640 1 -- 172.21.15.47:0/264741966 shutdown_connections 2024-09-15T21:56:55.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7b139a3640 1 -- 172.21.15.47:0/1087343832 >> 172.21.15.47:0/1087343832 conn(0x7f7b0c09eb90 msgr2=0x7f7b0c09efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7faa0cbfd640 1 --2- 172.21.15.47:0/264741966 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08098220 0x7faa08137650 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7faa0cbfd640 1 --2- 172.21.15.47:0/264741966 >> v2:172.21.15.47:3300/0 conn(0x7faa08097500 0x7faa080978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7faa0cbfd640 1 -- 172.21.15.47:0/264741966 >> 172.21.15.47:0/264741966 conn(0x7faa0809b880 msgr2=0x7faa0809bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7b139a3640 1 -- 172.21.15.47:0/1087343832 shutdown_connections 2024-09-15T21:56:55.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7b139a3640 1 -- 172.21.15.47:0/1087343832 wait complete. 2024-09-15T21:56:55.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7faa0cbfd640 1 -- 172.21.15.47:0/264741966 shutdown_connections 2024-09-15T21:56:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7faa0cbfd640 1 -- 172.21.15.47:0/264741966 wait complete. 2024-09-15T21:56:55.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7e97a0d640 1 -- 172.21.15.47:0/2006866445 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e90097420 msgr2=0x7f7e90094050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7e97a0d640 1 --2- 172.21.15.47:0/2006866445 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e90097420 0x7f7e90094050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e800097d0 tx=0x7f7e8002a000 comp rx=0 tx=0).stop 2024-09-15T21:56:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7e97a0d640 1 -- 172.21.15.47:0/2006866445 shutdown_connections 2024-09-15T21:56:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7e97a0d640 1 --2- 172.21.15.47:0/2006866445 >> v2:172.21.15.47:3300/0 conn(0x7f7e90094590 0x7f7e90094a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7e97a0d640 1 --2- 172.21.15.47:0/2006866445 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e90097420 0x7f7e90094050 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7e97a0d640 1 --2- 172.21.15.47:0/2006866445 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90096b00 0x7f7e90096ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7e97a0d640 1 -- 172.21.15.47:0/2006866445 >> 172.21.15.47:0/2006866445 conn(0x7f7e9009ddd0 msgr2=0x7f7e9009e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7e97a0d640 1 -- 172.21.15.47:0/2006866445 shutdown_connections 2024-09-15T21:56:55.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f7e97a0d640 1 -- 172.21.15.47:0/2006866445 wait complete. 2024-09-15T21:56:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.952+0000 7f9f06b2a640 1 Processor -- start 2024-09-15T21:56:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f9f06b2a640 1 -- start start 2024-09-15T21:56:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f7b139a3640 1 Processor -- start 2024-09-15T21:56:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f7b139a3640 1 -- start start 2024-09-15T21:56:55.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7faa0cbfd640 1 Processor -- start 2024-09-15T21:56:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f7e97a0d640 1 Processor -- start 2024-09-15T21:56:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7faa0cbfd640 1 -- start start 2024-09-15T21:56:55.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f7e97a0d640 1 -- start start 2024-09-15T21:56:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f9f06b2a640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0006dd00 0x7f9f00131030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f9f06b2a640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0006e890 0x7f9f00131570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f9f06b2a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f0012fbf0 0x7f9f00131ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f9f06b2a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9f00137d40 con 0x7f9f0012fbf0 2024-09-15T21:56:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f9f06b2a640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f9f0009c550 con 0x7f9f0006dd00 2024-09-15T21:56:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f9f06b2a640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f9f00137ec0 con 0x7f9f0006e890 2024-09-15T21:56:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f9f048c6640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0006dd00 0x7f9f00131030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.953+0000 7f9f048c6640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0006dd00 0x7f9f00131030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44816/0 (socket says 172.21.15.47:44816) 2024-09-15T21:56:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f9f048c6640 1 -- 172.21.15.47:0/1482437061 learned_addr learned my addr 172.21.15.47:0/1482437061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f9efffff640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0006e890 0x7f9f00131570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e97a0d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90094590 0x7f7e90128730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e97a0d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90096b00 0x7f7e90128c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b139a3640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09a630 0x7f7b0c131160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b139a3640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c09af50 0x7f7b0c1316a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b139a3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b0c12f920 0x7f7b0c134980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b139a3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c1368b0 con 0x7f7b0c12f920 2024-09-15T21:56:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7faa0cbfd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08097500 0x7faa0806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b139a3640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0c136730 con 0x7f7b0c09af50 2024-09-15T21:56:55.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7faa0cbfd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08097e20 0x7faa0806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e97a0d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e90097420 0x7f7e901291b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e97a0d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90097880 con 0x7f7e90096b00 2024-09-15T21:56:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e97a0d640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f7e90095340 con 0x7f7e90097420 2024-09-15T21:56:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e97a0d640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f7e900959f0 con 0x7f7e90094590 2024-09-15T21:56:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e94fa8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90096b00 0x7f7e90128c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e94fa8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90096b00 0x7f7e90128c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45956/0 (socket says 172.21.15.47:45956) 2024-09-15T21:56:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e94fa8640 1 -- 172.21.15.47:0/473676283 learned_addr learned my addr 172.21.15.47:0/473676283 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e957a9640 1 --2- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90094590 0x7f7e90128730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b139a3640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f7b0c136a30 con 0x7f7b0c09a630 2024-09-15T21:56:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7faa0cbfd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa08098220 0x7faa0806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f9f050c7640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f0012fbf0 0x7f9f00131ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7faa0cbfd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faa08137fa0 con 0x7faa08097500 2024-09-15T21:56:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f9f048c6640 1 -- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0006e890 msgr2=0x7f9f00131570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7faa0cbfd640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7faa08137e20 con 0x7faa08097e20 2024-09-15T21:56:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7faa05d9b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08097e20 0x7faa0806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f9f048c6640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0006e890 0x7f9f00131570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f9f048c6640 1 -- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f0012fbf0 msgr2=0x7f9f00131ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f9f048c6640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f0012fbf0 0x7f9f00131ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f9f048c6640 1 -- 172.21.15.47:0/1482437061 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f00132930 con 0x7f9f0006dd00 2024-09-15T21:56:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b10f3e640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c09af50 0x7f7b0c1316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b11f40640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b0c12f920 0x7f7b0c134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b11f40640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b0c12f920 0x7f7b0c134980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45972/0 (socket says 172.21.15.47:45972) 2024-09-15T21:56:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b11f40640 1 -- 172.21.15.47:0/3863715045 learned_addr learned my addr 172.21.15.47:0/3863715045 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7b1173f640 1 --2- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09a630 0x7f7b0c131160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7faa05d9b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08097e20 0x7faa0806f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44860/0 (socket says 172.21.15.47:44860) 2024-09-15T21:56:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7faa05d9b640 1 -- 172.21.15.47:0/2638697507 learned_addr learned my addr 172.21.15.47:0/2638697507 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7faa08138120 con 0x7faa08098220 2024-09-15T21:56:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7faa0659c640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08097500 0x7faa0806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e94fa8640 1 -- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90094590 msgr2=0x7f7e90128730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e94fa8640 1 --2- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90094590 0x7f7e90128730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e94fa8640 1 -- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e90097420 msgr2=0x7f7e901291b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e94fa8640 1 --2- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e90097420 0x7f7e901291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f7e94fa8640 1 -- 172.21.15.47:0/473676283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e90126570 con 0x7f7e90096b00 2024-09-15T21:56:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f9efffff640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0006e890 0x7f9f00131570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.954+0000 7f9f050c7640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f0012fbf0 0x7f9f00131ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7b10f3e640 1 -- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09a630 msgr2=0x7f7b0c131160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7b10f3e640 1 --2- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09a630 0x7f7b0c131160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7b10f3e640 1 -- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b0c12f920 msgr2=0x7f7b0c134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7b10f3e640 1 --2- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b0c12f920 0x7f7b0c134980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7b10f3e640 1 -- 172.21.15.47:0/3863715045 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c1333f0 con 0x7f7b0c09af50 2024-09-15T21:56:55.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7b11f40640 1 --2- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b0c12f920 0x7f7b0c134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f9f048c6640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0006dd00 0x7f9f00131030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4000c00 tx=0x7f9ef40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7b10f3e640 1 --2- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c09af50 0x7f7b0c1316a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c06d600 tx=0x7f7afc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7faa06d9d640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa08098220 0x7faa0806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7faa0659c640 1 -- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa08098220 msgr2=0x7faa0806f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7e94fa8640 1 --2- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90096b00 0x7f7e90128c70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7e80000c00 tx=0x7f7e800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f9efd7fa640 1 -- 172.21.15.47:0/1482437061 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4042070 con 0x7f9f0006dd00 2024-09-15T21:56:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7faa0659c640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa08098220 0x7faa0806f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7faa0659c640 1 -- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08097e20 msgr2=0x7faa0806f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7faa0659c640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08097e20 0x7faa0806f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7faa0659c640 1 -- 172.21.15.47:0/2638697507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa08072080 con 0x7faa08097500 2024-09-15T21:56:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f9efd7fa640 1 -- 172.21.15.47:0/1482437061 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ef4030770 con 0x7f9f0006dd00 2024-09-15T21:56:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7e85ffb640 1 -- 172.21.15.47:0/473676283 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80037040 con 0x7f7e90096b00 2024-09-15T21:56:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7e85ffb640 1 -- 172.21.15.47:0/473676283 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80042070 con 0x7f7e90096b00 2024-09-15T21:56:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7e85ffb640 1 -- 172.21.15.47:0/473676283 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8003b890 con 0x7f7e90096b00 2024-09-15T21:56:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7faa05d9b640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08097e20 0x7faa0806f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7faa06d9d640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa08098220 0x7faa0806f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:56:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7b01ffb640 1 -- 172.21.15.47:0/3863715045 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc037040 con 0x7f7b0c09af50 2024-09-15T21:56:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7faa0659c640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08097500 0x7faa0806ed20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800b780 tx=0x7fa9f800bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:55.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7e97a0d640 1 -- 172.21.15.47:0/473676283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e90127730 con 0x7f7e90096b00 2024-09-15T21:56:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7b01ffb640 1 -- 172.21.15.47:0/3863715045 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7afc02ab20 con 0x7f7b0c09af50 2024-09-15T21:56:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f7b139a3640 1 -- 172.21.15.47:0/3863715045 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0c131be0 con 0x7f7b0c09af50 2024-09-15T21:56:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.956+0000 7f7b139a3640 1 -- 172.21.15.47:0/3863715045 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0c133cf0 con 0x7f7b0c09af50 2024-09-15T21:56:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.956+0000 7fa9f6ffd640 1 -- 172.21.15.47:0/2638697507 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8019070 con 0x7faa08097500 2024-09-15T21:56:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.956+0000 7fa9f6ffd640 1 -- 172.21.15.47:0/2638697507 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f800c720 con 0x7faa08097500 2024-09-15T21:56:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.956+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa08071030 con 0x7faa08097500 2024-09-15T21:56:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.956+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa080726e0 con 0x7faa08097500 2024-09-15T21:56:55.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.956+0000 7fa9f6ffd640 1 -- 172.21.15.47:0/2638697507 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800cb10 con 0x7faa08097500 2024-09-15T21:56:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f9efd7fa640 1 -- 172.21.15.47:0/1482437061 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4037070 con 0x7f9f0006dd00 2024-09-15T21:56:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f9f06b2a640 1 -- 172.21.15.47:0/1482437061 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f001341f0 con 0x7f9f0006dd00 2024-09-15T21:56:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.955+0000 7f9f06b2a640 1 -- 172.21.15.47:0/1482437061 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f00133530 con 0x7f9f0006dd00 2024-09-15T21:56:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.956+0000 7f7b01ffb640 1 -- 172.21.15.47:0/3863715045 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc037040 con 0x7f7b0c09af50 2024-09-15T21:56:55.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.956+0000 7f7e97a0d640 1 -- 172.21.15.47:0/473676283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e90124c70 con 0x7f7e90096b00 2024-09-15T21:56:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.957+0000 7f9f06b2a640 1 -- 172.21.15.47:0/1482437061 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f0012ffd0 con 0x7f9f0006dd00 2024-09-15T21:56:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.957+0000 7f7e737fe640 1 -- 172.21.15.47:0/473676283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6c005290 con 0x7f7e90096b00 2024-09-15T21:56:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.957+0000 7f9efd7fa640 1 -- 172.21.15.47:0/1482437061 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f9ef4045070 con 0x7f9f0006dd00 2024-09-15T21:56:55.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.958+0000 7f7b139a3640 1 -- 172.21.15.47:0/3863715045 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b0c132a30 con 0x7f7b0c09af50 2024-09-15T21:56:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.958+0000 7f7e85ffb640 1 -- 172.21.15.47:0/473676283 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f7e8003ba30 con 0x7f7e90096b00 2024-09-15T21:56:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.958+0000 7f9efd7fa640 1 -- 172.21.15.47:0/1482437061 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f9ef407de80 con 0x7f9f0006dd00 2024-09-15T21:56:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.958+0000 7f7e85ffb640 1 -- 172.21.15.47:0/473676283 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f7e80082d60 con 0x7f7e90096b00 2024-09-15T21:56:55.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.958+0000 7f7b01ffb640 1 -- 172.21.15.47:0/3863715045 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f7afc044020 con 0x7f7b0c09af50 2024-09-15T21:56:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.958+0000 7fa9f6ffd640 1 -- 172.21.15.47:0/2638697507 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fa9f801c070 con 0x7faa08097500 2024-09-15T21:56:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.959+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ec005290 con 0x7faa08097500 2024-09-15T21:56:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.959+0000 7f7b01ffb640 1 -- 172.21.15.47:0/3863715045 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f7afc02acd0 con 0x7f7b0c09af50 2024-09-15T21:56:55.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.959+0000 7fa9f6ffd640 1 -- 172.21.15.47:0/2638697507 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fa9f805ce40 con 0x7faa08097500 2024-09-15T21:56:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.962+0000 7f7e85ffb640 1 -- 172.21.15.47:0/473676283 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f7e8007cfa0 con 0x7f7e90096b00 2024-09-15T21:56:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.962+0000 7f9efd7fa640 1 -- 172.21.15.47:0/1482437061 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f9ef4049940 con 0x7f9f0006dd00 2024-09-15T21:56:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.966+0000 7fa9f6ffd640 1 -- 172.21.15.47:0/2638697507 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fa9f803c010 con 0x7faa08097500 2024-09-15T21:56:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:55.966+0000 7f7b01ffb640 1 -- 172.21.15.47:0/3863715045 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f7afc003d40 con 0x7f7b0c09af50 2024-09-15T21:56:56.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.112+0000 7f7e737fe640 1 -- 172.21.15.47:0/473676283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7e6c005580 con 0x7f7e90096b00 2024-09-15T21:56:56.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.113+0000 7f7e85ffb640 1 -- 172.21.15.47:0/473676283 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7e8005e080 con 0x7f7e90096b00 2024-09-15T21:56:56.115 INFO:teuthology.orchestra.run.smithi047.stdout:115964116994 2024-09-15T21:56:56.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.114+0000 7f9f06b2a640 1 -- 172.21.15.47:0/1482437061 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9f0006f400 con 0x7f9f0006dd00 2024-09-15T21:56:56.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.114+0000 7f9efd7fa640 1 -- 172.21.15.47:0/1482437061 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9ef4049ba0 con 0x7f9f0006dd00 2024-09-15T21:56:56.117 INFO:teuthology.orchestra.run.smithi047.stdout:107374182402 2024-09-15T21:56:56.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.115+0000 7f7e737fe640 1 -- 172.21.15.47:0/473676283 >> v1:172.21.15.47:6800/1790418198 conn(0x7f7e74048020 legacy=0x7f7e7404a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:56.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.115+0000 7f7e737fe640 1 -- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90096b00 msgr2=0x7f7e90128c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.115+0000 7f7e737fe640 1 --2- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90096b00 0x7f7e90128c70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7e80000c00 tx=0x7f7e800046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.115+0000 7f7e737fe640 1 -- 172.21.15.47:0/473676283 shutdown_connections 2024-09-15T21:56:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.115+0000 7f7e737fe640 1 --2- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e90097420 0x7f7e901291b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.115+0000 7f7e737fe640 1 --2- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e90096b00 0x7f7e90128c70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.115+0000 7f7e737fe640 1 --2- 172.21.15.47:0/473676283 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e90094590 0x7f7e90128730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.115+0000 7f7e737fe640 1 -- 172.21.15.47:0/473676283 >> 172.21.15.47:0/473676283 conn(0x7f7e9009ddd0 msgr2=0x7f7e90099480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.115+0000 7f7e737fe640 1 -- 172.21.15.47:0/473676283 shutdown_connections 2024-09-15T21:56:56.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.115+0000 7f7e737fe640 1 -- 172.21.15.47:0/473676283 wait complete. 2024-09-15T21:56:56.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.116+0000 7f7b139a3640 1 -- 172.21.15.47:0/3863715045 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7b0c12f3e0 con 0x7f7b0c09af50 2024-09-15T21:56:56.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.116+0000 7f7b01ffb640 1 -- 172.21.15.47:0/3863715045 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7afc003d40 con 0x7f7b0c09af50 2024-09-15T21:56:56.119 INFO:teuthology.orchestra.run.smithi047.stdout:115964116994 2024-09-15T21:56:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.117+0000 7f9f06b2a640 1 -- 172.21.15.47:0/1482437061 >> v1:172.21.15.47:6800/1790418198 conn(0x7f9ee4048000 legacy=0x7f9ee404a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.117+0000 7f9f06b2a640 1 -- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0006dd00 msgr2=0x7f9f00131030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:56.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.117+0000 7f9f06b2a640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0006dd00 0x7f9f00131030 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4000c00 tx=0x7f9ef40046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.117+0000 7f9f06b2a640 1 -- 172.21.15.47:0/1482437061 shutdown_connections 2024-09-15T21:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.117+0000 7f9f06b2a640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f0012fbf0 0x7f9f00131ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.117+0000 7f9f06b2a640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f0006e890 0x7f9f00131570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.117+0000 7f9f06b2a640 1 --2- 172.21.15.47:0/1482437061 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f0006dd00 0x7f9f00131030 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.117+0000 7f9f06b2a640 1 -- 172.21.15.47:0/1482437061 >> 172.21.15.47:0/1482437061 conn(0x7f9f0009ece0 msgr2=0x7f9f0012eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.117+0000 7f9f06b2a640 1 -- 172.21.15.47:0/1482437061 shutdown_connections 2024-09-15T21:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.117+0000 7f9f06b2a640 1 -- 172.21.15.47:0/1482437061 wait complete. 2024-09-15T21:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.118+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa9ec005580 con 0x7faa08097500 2024-09-15T21:56:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.118+0000 7fa9f6ffd640 1 -- 172.21.15.47:0/2638697507 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa9f803c270 con 0x7faa08097500 2024-09-15T21:56:56.122 INFO:teuthology.orchestra.run.smithi047.stdout:107374182402 2024-09-15T21:56:56.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.118+0000 7f7b139a3640 1 -- 172.21.15.47:0/3863715045 >> v1:172.21.15.47:6800/1790418198 conn(0x7f7af00481d0 legacy=0x7f7af004a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.118+0000 7f7b139a3640 1 -- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c09af50 msgr2=0x7f7b0c1316a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.118+0000 7f7b139a3640 1 --2- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c09af50 0x7f7b0c1316a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c06d600 tx=0x7f7afc0046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.119+0000 7f7b139a3640 1 -- 172.21.15.47:0/3863715045 shutdown_connections 2024-09-15T21:56:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.119+0000 7f7b139a3640 1 --2- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b0c12f920 0x7f7b0c134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.119+0000 7f7b139a3640 1 --2- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7b0c09af50 0x7f7b0c1316a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.119+0000 7f7b139a3640 1 --2- 172.21.15.47:0/3863715045 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7b0c09a630 0x7f7b0c131160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.119+0000 7f7b139a3640 1 -- 172.21.15.47:0/3863715045 >> 172.21.15.47:0/3863715045 conn(0x7f7b0c09eb90 msgr2=0x7f7b0c12e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.119+0000 7f7b139a3640 1 -- 172.21.15.47:0/3863715045 shutdown_connections 2024-09-15T21:56:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.119+0000 7f7b139a3640 1 -- 172.21.15.47:0/3863715045 wait complete. 2024-09-15T21:56:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.120+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 >> v1:172.21.15.47:6800/1790418198 conn(0x7fa9e80480a0 legacy=0x7fa9e804a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.120+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08097500 msgr2=0x7faa0806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.120+0000 7faa0cbfd640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08097500 0x7faa0806ed20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800b780 tx=0x7fa9f800bc40 comp rx=0 tx=0).stop 2024-09-15T21:56:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.120+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 shutdown_connections 2024-09-15T21:56:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.120+0000 7faa0cbfd640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7faa08098220 0x7faa0806f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.120+0000 7faa0cbfd640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7faa08097e20 0x7faa0806f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.120+0000 7faa0cbfd640 1 --2- 172.21.15.47:0/2638697507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa08097500 0x7faa0806ed20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.120+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 >> 172.21.15.47:0/2638697507 conn(0x7faa0809b880 msgr2=0x7faa08132360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.121+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 shutdown_connections 2024-09-15T21:56:56.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:56.121+0000 7faa0cbfd640 1 -- 172.21.15.47:0/2638697507 wait complete. 2024-09-15T21:56:56.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116994 for osd.3 2024-09-15T21:56:56.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116994 for osd.2 2024-09-15T21:56:56.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182404 got 107374182402 for osd.1 2024-09-15T21:56:56.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182404 got 107374182402 for osd.0 2024-09-15T21:56:57.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:56:57.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:56:57.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:56:57.138 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:56:57.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.254+0000 7f47b02d7640 1 Processor -- start 2024-09-15T21:56:57.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7f47b02d7640 1 -- start start 2024-09-15T21:56:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7f4a09e55640 1 Processor -- start 2024-09-15T21:56:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7f4a09e55640 1 -- start start 2024-09-15T21:56:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7fce90c08640 1 Processor -- start 2024-09-15T21:56:57.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7fce90c08640 1 -- start start 2024-09-15T21:56:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7f47b02d7640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812e770 0x7f47a812eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7f47b02d7640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a812f090 0x7f47a812f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7f47b02d7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47a8097030 0x7f47a812fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7f47b02d7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f47a806bdd0 con 0x7f47a8097030 2024-09-15T21:56:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7f47b02d7640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f47a8068410 con 0x7f47a812f090 2024-09-15T21:56:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.255+0000 7f47b02d7640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f47a8097b00 con 0x7f47a812e770 2024-09-15T21:56:57.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f4a09e55640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a0409b120 0x7f4a04134720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce90c08640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12a1f0 0x7fce8c12a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce90c08640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fce8c12ab10 0x7fce8c12d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce90c08640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 0x7fce8c12dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce90c08640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fce8c09f9d0 con 0x7fce8c12ab10 2024-09-15T21:56:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f4a037fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4a0409a630 0x7f4a0409aa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34384/0 (socket says 172.21.15.47:34384) 2024-09-15T21:56:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f4a037fe640 1 -- 172.21.15.47:0/1273903219 learned_addr learned my addr 172.21.15.47:0/1273903219 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f4a09e55640 1 --2- 172.21.15.47:0/1273903219 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a04134c60 0x7f4a04137050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce90c08640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fce8c099990 con 0x7fce8c12a1f0 2024-09-15T21:56:57.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce90c08640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fce8c0995c0 con 0x7fce8c12d8b0 2024-09-15T21:56:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f4a09e55640 1 -- 172.21.15.47:0/1273903219 --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0409b600 con 0x7f4a0409a630 2024-09-15T21:56:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f4a09e55640 1 -- 172.21.15.47:0/1273903219 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0406d480 con 0x7f4a04134c60 2024-09-15T21:56:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f4a09e55640 1 -- 172.21.15.47:0/1273903219 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f4a0406d140 con 0x7f4a0409b120 2024-09-15T21:56:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce89d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fce8c12ab10 0x7fce8c12d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce89d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fce8c12ab10 0x7fce8c12d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46000/0 (socket says 172.21.15.47:46000) 2024-09-15T21:56:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce89d9b640 1 -- 172.21.15.47:0/561302733 learned_addr learned my addr 172.21.15.47:0/561302733 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:57.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce8a59c640 1 --2- 172.21.15.47:0/561302733 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12a1f0 0x7fce8c12a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f4a03fff640 1 --2- 172.21.15.47:0/1273903219 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a04134c60 0x7f4a04137050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce8ad9d640 1 --2- 172.21.15.47:0/561302733 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 0x7fce8c12dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f4a02ffd640 1 --2- 172.21.15.47:0/1273903219 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a0409b120 0x7f4a04134720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f47ae874640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47a8097030 0x7f47a812fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7fce8a59c640 1 -- 172.21.15.47:0/561302733 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 msgr2=0x7fce8c12dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:57.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f47ae874640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47a8097030 0x7f47a812fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45996/0 (socket says 172.21.15.47:45996) 2024-09-15T21:56:57.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7f4a02ffd640 1 -- 172.21.15.47:0/1273903219 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a04134c60 msgr2=0x7f4a04137050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:57.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7f4a02ffd640 1 --2- 172.21.15.47:0/1273903219 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a04134c60 0x7f4a04137050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7f4a02ffd640 1 -- 172.21.15.47:0/1273903219 >> v1:172.21.15.47:6789/0 conn(0x7f4a0409a630 legacy=0x7f4a0409aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7f4a02ffd640 1 -- 172.21.15.47:0/1273903219 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0409bc00 con 0x7f4a0409b120 2024-09-15T21:56:57.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f47ae874640 1 -- 172.21.15.47:0/572714579 learned_addr learned my addr 172.21.15.47:0/572714579 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:57.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f47ad872640 1 --2- 172.21.15.47:0/572714579 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a812f090 0x7f47a812f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.256+0000 7f47ae073640 1 --2- 172.21.15.47:0/572714579 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812e770 0x7f47a812eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7f47ae073640 1 -- 172.21.15.47:0/572714579 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a812f090 msgr2=0x7f47a812f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:57.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7f47ae073640 1 --2- 172.21.15.47:0/572714579 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a812f090 0x7f47a812f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7f47ae073640 1 -- 172.21.15.47:0/572714579 >> v2:172.21.15.47:3300/0 conn(0x7f47a8097030 msgr2=0x7f47a812fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:57.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7f47ae073640 1 --2- 172.21.15.47:0/572714579 >> v2:172.21.15.47:3300/0 conn(0x7f47a8097030 0x7f47a812fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7f47ae073640 1 -- 172.21.15.47:0/572714579 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a812ff90 con 0x7f47a812e770 2024-09-15T21:56:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7fce8a59c640 1 --2- 172.21.15.47:0/561302733 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 0x7fce8c12dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7fce8a59c640 1 -- 172.21.15.47:0/561302733 >> v2:172.21.15.47:3300/0 conn(0x7fce8c12ab10 msgr2=0x7fce8c12d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7fce8a59c640 1 --2- 172.21.15.47:0/561302733 >> v2:172.21.15.47:3300/0 conn(0x7fce8c12ab10 0x7fce8c12d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7fce8a59c640 1 -- 172.21.15.47:0/561302733 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8c099ec0 con 0x7fce8c12a1f0 2024-09-15T21:56:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7fce89d9b640 1 --2- 172.21.15.47:0/561302733 >> v2:172.21.15.47:3300/0 conn(0x7fce8c12ab10 0x7fce8c12d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.257+0000 7fce8ad9d640 1 --2- 172.21.15.47:0/561302733 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 0x7fce8c12dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:57.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.258+0000 7f4a02ffd640 1 --2- 172.21.15.47:0/1273903219 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a0409b120 0x7f4a04134720 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49f80097d0 tx=0x7f49f802a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ef6434997a7bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:57.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.258+0000 7fce8a59c640 1 --2- 172.21.15.47:0/561302733 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12a1f0 0x7fce8c12a5d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009800 tx=0x7fce7c02a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e583cf12cb4c7c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:57.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.258+0000 7f4a027fc640 1 -- 172.21.15.47:0/1273903219 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8037070 con 0x7f4a0409b120 2024-09-15T21:56:57.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.258+0000 7f47ae073640 1 --2- 172.21.15.47:0/572714579 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812e770 0x7f47a812eb50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47a40097a0 tx=0x7f47a4029fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c5a5a8d6e74f423 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:57.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.258+0000 7f4a027fc640 1 -- 172.21.15.47:0/1273903219 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f802ad90 con 0x7f4a0409b120 2024-09-15T21:56:57.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.258+0000 7f4a027fc640 1 -- 172.21.15.47:0/1273903219 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8032660 con 0x7f4a0409b120 2024-09-15T21:56:57.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.258+0000 7fce8959a640 1 -- 172.21.15.47:0/561302733 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c037070 con 0x7fce8c12a1f0 2024-09-15T21:56:57.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.258+0000 7fce8959a640 1 -- 172.21.15.47:0/561302733 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce7c02adc0 con 0x7fce8c12a1f0 2024-09-15T21:56:57.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.258+0000 7fce8959a640 1 -- 172.21.15.47:0/561302733 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c032660 con 0x7fce8c12a1f0 2024-09-15T21:56:57.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f47ad071640 1 -- 172.21.15.47:0/572714579 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4037070 con 0x7f47a812e770 2024-09-15T21:56:57.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f47ad071640 1 -- 172.21.15.47:0/572714579 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a402ad20 con 0x7f47a812e770 2024-09-15T21:56:57.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f47ad071640 1 -- 172.21.15.47:0/572714579 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4032660 con 0x7f47a812e770 2024-09-15T21:56:57.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f4a09e55640 1 -- 172.21.15.47:0/1273903219 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a0409b120 msgr2=0x7f4a04134720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7fce90c08640 1 -- 172.21.15.47:0/561302733 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12a1f0 msgr2=0x7fce8c12a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f4a09e55640 1 --2- 172.21.15.47:0/1273903219 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a0409b120 0x7f4a04134720 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f49f80097d0 tx=0x7f49f802a040 comp rx=0 tx=0).stop 2024-09-15T21:56:57.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7fce90c08640 1 --2- 172.21.15.47:0/561302733 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12a1f0 0x7fce8c12a5d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009800 tx=0x7fce7c02a0b0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f4a09e55640 1 -- 172.21.15.47:0/1273903219 shutdown_connections 2024-09-15T21:56:57.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f4a09e55640 1 --2- 172.21.15.47:0/1273903219 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a04134c60 0x7f4a04137050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f4a09e55640 1 --2- 172.21.15.47:0/1273903219 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a0409b120 0x7f4a04134720 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f4a09e55640 1 -- 172.21.15.47:0/1273903219 >> 172.21.15.47:0/1273903219 conn(0x7f4a0409eb90 msgr2=0x7f4a0409efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:57.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7fce90c08640 1 -- 172.21.15.47:0/561302733 shutdown_connections 2024-09-15T21:56:57.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7fce90c08640 1 --2- 172.21.15.47:0/561302733 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 0x7fce8c12dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7fce90c08640 1 --2- 172.21.15.47:0/561302733 >> v2:172.21.15.47:3300/0 conn(0x7fce8c12ab10 0x7fce8c12d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7fce90c08640 1 --2- 172.21.15.47:0/561302733 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12a1f0 0x7fce8c12a5d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7fce90c08640 1 -- 172.21.15.47:0/561302733 >> 172.21.15.47:0/561302733 conn(0x7fce8c09db80 msgr2=0x7fce8c09df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:57.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f4a09e55640 1 -- 172.21.15.47:0/1273903219 shutdown_connections 2024-09-15T21:56:57.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f4a09e55640 1 -- 172.21.15.47:0/1273903219 wait complete. 2024-09-15T21:56:57.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7fce90c08640 1 -- 172.21.15.47:0/561302733 shutdown_connections 2024-09-15T21:56:57.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7fce90c08640 1 -- 172.21.15.47:0/561302733 wait complete. 2024-09-15T21:56:57.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f47b02d7640 1 -- 172.21.15.47:0/572714579 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812e770 msgr2=0x7f47a812eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.259+0000 7f47b02d7640 1 --2- 172.21.15.47:0/572714579 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812e770 0x7f47a812eb50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47a40097a0 tx=0x7f47a4029fd0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.260+0000 7f47b02d7640 1 -- 172.21.15.47:0/572714579 shutdown_connections 2024-09-15T21:56:57.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.260+0000 7f47b02d7640 1 --2- 172.21.15.47:0/572714579 >> v2:172.21.15.47:3300/0 conn(0x7f47a8097030 0x7f47a812fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.260+0000 7f47b02d7640 1 --2- 172.21.15.47:0/572714579 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a812f090 0x7f47a812f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.260+0000 7f47b02d7640 1 --2- 172.21.15.47:0/572714579 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812e770 0x7f47a812eb50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.260+0000 7f47b02d7640 1 -- 172.21.15.47:0/572714579 >> 172.21.15.47:0/572714579 conn(0x7f47a809c5d0 msgr2=0x7f47a809c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:57.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.260+0000 7f47b02d7640 1 -- 172.21.15.47:0/572714579 shutdown_connections 2024-09-15T21:56:57.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.260+0000 7f47b02d7640 1 -- 172.21.15.47:0/572714579 wait complete. 2024-09-15T21:56:57.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.260+0000 7f4a09e55640 1 Processor -- start 2024-09-15T21:56:57.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.260+0000 7fce90c08640 1 Processor -- start 2024-09-15T21:56:57.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce90c08640 1 -- start start 2024-09-15T21:56:57.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7f4a09e55640 1 -- start start 2024-09-15T21:56:57.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7f47b02d7640 1 Processor -- start 2024-09-15T21:56:57.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce90c08640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8c12a1f0 0x7fce8c126140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce90c08640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12ab10 0x7fce8c126680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7f47b02d7640 1 -- start start 2024-09-15T21:56:57.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce90c08640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 0x7fce8c126bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7f47b02d7640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a8097030 0x7f47a812d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47b02d7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47a812e770 0x7f47a812dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7f4a09e55640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a0409a630 0x7f4a04126f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7f4a09e55640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a0409b120 0x7f4a041274b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7f4a09e55640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a04134c60 0x7f4a041279f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7f4a09e55640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4a04137920 con 0x7f4a0409a630 2024-09-15T21:56:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7f4a09e55640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f4a041377a0 con 0x7f4a0409b120 2024-09-15T21:56:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7f4a09e55640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f4a04137aa0 con 0x7f4a04134c60 2024-09-15T21:56:57.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a02ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a0409b120 0x7f4a041274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce90c08640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c06dfc0 con 0x7fce8c12a1f0 2024-09-15T21:56:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce90c08640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fce8c06de40 con 0x7fce8c12ab10 2024-09-15T21:56:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce90c08640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fce8c06e140 con 0x7fce8c12d8b0 2024-09-15T21:56:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce89d9b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12ab10 0x7fce8c126680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce8a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8c12a1f0 0x7fce8c126140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce89d9b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12ab10 0x7fce8c126680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44906/0 (socket says 172.21.15.47:44906) 2024-09-15T21:56:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce89d9b640 1 -- 172.21.15.47:0/4036288287 learned_addr learned my addr 172.21.15.47:0/4036288287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.261+0000 7fce8ad9d640 1 --2- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 0x7fce8c126bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47b02d7640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812f090 0x7f47a806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47b02d7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8136bd0 con 0x7f47a812e770 2024-09-15T21:56:57.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a02ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a0409b120 0x7f4a041274b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44914/0 (socket says 172.21.15.47:44914) 2024-09-15T21:56:57.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a02ffd640 1 -- 172.21.15.47:0/2781669495 learned_addr learned my addr 172.21.15.47:0/2781669495 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:57.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a03fff640 1 --2- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a04134c60 0x7f4a041279f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47b02d7640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f47a8136a50 con 0x7f47a8097030 2024-09-15T21:56:57.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47b02d7640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f47a8136d50 con 0x7f47a812f090 2024-09-15T21:56:57.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a037fe640 1 --2- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a0409a630 0x7f4a04126f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7fce89d9b640 1 -- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 msgr2=0x7fce8c126bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7fce89d9b640 1 --2- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 0x7fce8c126bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7fce89d9b640 1 -- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8c12a1f0 msgr2=0x7fce8c126140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7fce89d9b640 1 --2- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8c12a1f0 0x7fce8c126140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7fce89d9b640 1 -- 172.21.15.47:0/4036288287 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8c129570 con 0x7fce8c12ab10 2024-09-15T21:56:57.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47ae073640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a8097030 0x7f47a812d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47ae073640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a8097030 0x7f47a812d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44918/0 (socket says 172.21.15.47:44918) 2024-09-15T21:56:57.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47ae073640 1 -- 172.21.15.47:0/934229667 learned_addr learned my addr 172.21.15.47:0/934229667 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:57.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47ad872640 1 --2- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47a812e770 0x7f47a812dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47ae874640 1 --2- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812f090 0x7f47a806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a03fff640 1 -- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a0409b120 msgr2=0x7f4a041274b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a03fff640 1 --2- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a0409b120 0x7f4a041274b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a03fff640 1 -- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a0409a630 msgr2=0x7f4a04126f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a03fff640 1 --2- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a0409a630 0x7f4a04126f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a03fff640 1 -- 172.21.15.47:0/2781669495 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a04129320 con 0x7f4a04134c60 2024-09-15T21:56:57.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7fce89d9b640 1 --2- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12ab10 0x7fce8c126680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce80006e00 tx=0x7fce8000ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:57.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a037fe640 1 --2- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a0409a630 0x7f4a04126f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:57.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47ae073640 1 -- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812f090 msgr2=0x7f47a806e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7fce7affd640 1 -- 172.21.15.47:0/4036288287 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8000ca00 con 0x7fce8c12ab10 2024-09-15T21:56:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47ae073640 1 --2- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812f090 0x7f47a806e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47ae073640 1 -- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47a812e770 msgr2=0x7f47a812dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47ae073640 1 --2- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47a812e770 0x7f47a812dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f47ae073640 1 -- 172.21.15.47:0/934229667 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a80700f0 con 0x7f47a8097030 2024-09-15T21:56:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7f47ae073640 1 --2- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a8097030 0x7f47a812d680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f47a40098c0 tx=0x7f47a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.262+0000 7f4a03fff640 1 --2- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a04134c60 0x7f4a041279f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00ef90 tx=0x7f49fc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7fce7affd640 1 -- 172.21.15.47:0/4036288287 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce8000cba0 con 0x7fce8c12ab10 2024-09-15T21:56:57.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7fce90c08640 1 -- 172.21.15.47:0/4036288287 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8c128df0 con 0x7fce8c12ab10 2024-09-15T21:56:57.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7f479effd640 1 -- 172.21.15.47:0/934229667 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4042020 con 0x7f47a8097030 2024-09-15T21:56:57.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7f479effd640 1 -- 172.21.15.47:0/934229667 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a403d690 con 0x7f47a8097030 2024-09-15T21:56:57.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7f479effd640 1 -- 172.21.15.47:0/934229667 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4037040 con 0x7f47a8097030 2024-09-15T21:56:57.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7f49ebfff640 1 -- 172.21.15.47:0/2781669495 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc018020 con 0x7f4a04134c60 2024-09-15T21:56:57.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7f49ebfff640 1 -- 172.21.15.47:0/2781669495 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49fc014480 con 0x7f4a04134c60 2024-09-15T21:56:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7f49ebfff640 1 -- 172.21.15.47:0/2781669495 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc014870 con 0x7f4a04134c60 2024-09-15T21:56:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7f4a09e55640 1 -- 172.21.15.47:0/2781669495 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a041281d0 con 0x7f4a04134c60 2024-09-15T21:56:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7f4a09e55640 1 -- 172.21.15.47:0/2781669495 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a04129590 con 0x7f4a04134c60 2024-09-15T21:56:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7fce90c08640 1 -- 172.21.15.47:0/4036288287 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8c1c8d40 con 0x7fce8c12ab10 2024-09-15T21:56:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.263+0000 7f47b02d7640 1 -- 172.21.15.47:0/934229667 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a806fa30 con 0x7f47a8097030 2024-09-15T21:56:57.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.264+0000 7fce7affd640 1 -- 172.21.15.47:0/4036288287 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8000ca00 con 0x7fce8c12ab10 2024-09-15T21:56:57.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.264+0000 7f47b02d7640 1 -- 172.21.15.47:0/934229667 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a8070dd0 con 0x7f47a8097030 2024-09-15T21:56:57.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.264+0000 7f4a09e55640 1 -- 172.21.15.47:0/2781669495 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0409aa10 con 0x7f4a04134c60 2024-09-15T21:56:57.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.265+0000 7fce90c08640 1 -- 172.21.15.47:0/4036288287 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce8c073eb0 con 0x7fce8c12ab10 2024-09-15T21:56:57.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.265+0000 7f49ebfff640 1 -- 172.21.15.47:0/2781669495 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f49fc005d10 con 0x7f4a04134c60 2024-09-15T21:56:57.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.265+0000 7f479effd640 1 -- 172.21.15.47:0/934229667 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f47a4045020 con 0x7f47a8097030 2024-09-15T21:56:57.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.265+0000 7f49ebfff640 1 -- 172.21.15.47:0/2781669495 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f49fc05e650 con 0x7f4a04134c60 2024-09-15T21:56:57.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.265+0000 7f47b02d7640 1 -- 172.21.15.47:0/934229667 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a809de00 con 0x7f47a8097030 2024-09-15T21:56:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.265+0000 7fce7affd640 1 -- 172.21.15.47:0/4036288287 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fce80004030 con 0x7fce8c12ab10 2024-09-15T21:56:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.266+0000 7f479effd640 1 -- 172.21.15.47:0/934229667 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f47a407e120 con 0x7f47a8097030 2024-09-15T21:56:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.266+0000 7fce7affd640 1 -- 172.21.15.47:0/4036288287 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fce8000f040 con 0x7fce8c12ab10 2024-09-15T21:56:57.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.270+0000 7f479effd640 1 -- 172.21.15.47:0/934229667 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f47a809de00 con 0x7f47a8097030 2024-09-15T21:56:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.272+0000 7f49ebfff640 1 -- 172.21.15.47:0/2781669495 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f49fc022030 con 0x7f4a04134c60 2024-09-15T21:56:57.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.276+0000 7fce7affd640 1 -- 172.21.15.47:0/4036288287 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fce8c073eb0 con 0x7fce8c12ab10 2024-09-15T21:56:57.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.297+0000 7f9d5e296640 1 Processor -- start 2024-09-15T21:56:57.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.297+0000 7f9d5e296640 1 -- start start 2024-09-15T21:56:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.298+0000 7f9d5e296640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809a630 0x7f9d5809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.298+0000 7f9d5e296640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5812faf0 0x7f9d5812ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.298+0000 7f9d5e296640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5809b600 con 0x7f9d5809af50 2024-09-15T21:56:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.298+0000 7f9d5e296640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f9d5806d480 con 0x7f9d5809a630 2024-09-15T21:56:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.298+0000 7f9d5e296640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f9d5806d140 con 0x7f9d5812faf0 2024-09-15T21:56:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.298+0000 7f9d577fe640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9d5809af50 0x7f9d5812f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34392/0 (socket says 172.21.15.47:34392) 2024-09-15T21:56:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.298+0000 7f9d577fe640 1 -- 172.21.15.47:0/1464035527 learned_addr learned my addr 172.21.15.47:0/1464035527 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.298+0000 7f9d5c833640 1 --2- 172.21.15.47:0/1464035527 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5812faf0 0x7f9d5812ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.298+0000 7f9d57fff640 1 --2- 172.21.15.47:0/1464035527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809a630 0x7f9d5809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.299+0000 7f9d5c833640 1 -- 172.21.15.47:0/1464035527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809a630 msgr2=0x7f9d5809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:57.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.299+0000 7f9d5c833640 1 --2- 172.21.15.47:0/1464035527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809a630 0x7f9d5809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.299+0000 7f9d5c833640 1 -- 172.21.15.47:0/1464035527 >> v1:172.21.15.47:6789/0 conn(0x7f9d5809af50 legacy=0x7f9d5812f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.299+0000 7f9d5c833640 1 -- 172.21.15.47:0/1464035527 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5809bc00 con 0x7f9d5812faf0 2024-09-15T21:56:57.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.300+0000 7f9d5c833640 1 --2- 172.21.15.47:0/1464035527 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5812faf0 0x7f9d5812ff80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009dc0 tx=0x7f9d5002a510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97db7ab6353f4391 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.300+0000 7f9d56ffd640 1 -- 172.21.15.47:0/1464035527 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50007740 con 0x7f9d5812faf0 2024-09-15T21:56:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.300+0000 7f9d56ffd640 1 -- 172.21.15.47:0/1464035527 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d500078e0 con 0x7f9d5812faf0 2024-09-15T21:56:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.300+0000 7f9d56ffd640 1 -- 172.21.15.47:0/1464035527 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50007cb0 con 0x7f9d5812faf0 2024-09-15T21:56:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.301+0000 7f9d5e296640 1 -- 172.21.15.47:0/1464035527 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5812faf0 msgr2=0x7f9d5812ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.301+0000 7f9d5e296640 1 --2- 172.21.15.47:0/1464035527 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5812faf0 0x7f9d5812ff80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009dc0 tx=0x7f9d5002a510 comp rx=0 tx=0).stop 2024-09-15T21:56:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.301+0000 7f9d5e296640 1 -- 172.21.15.47:0/1464035527 shutdown_connections 2024-09-15T21:56:57.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.301+0000 7f9d5e296640 1 --2- 172.21.15.47:0/1464035527 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5812faf0 0x7f9d5812ff80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.301+0000 7f9d5e296640 1 --2- 172.21.15.47:0/1464035527 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809a630 0x7f9d5809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.301+0000 7f9d5e296640 1 -- 172.21.15.47:0/1464035527 >> 172.21.15.47:0/1464035527 conn(0x7f9d5809eb90 msgr2=0x7f9d5809efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.301+0000 7f9d5e296640 1 -- 172.21.15.47:0/1464035527 shutdown_connections 2024-09-15T21:56:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.301+0000 7f9d5e296640 1 -- 172.21.15.47:0/1464035527 wait complete. 2024-09-15T21:56:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.302+0000 7f9d5e296640 1 Processor -- start 2024-09-15T21:56:57.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.302+0000 7f9d5e296640 1 -- start start 2024-09-15T21:56:57.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.302+0000 7f9d5e296640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5809a630 0x7f9d58130770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.302+0000 7f9d5e296640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809af50 0x7f9d58130cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.302+0000 7f9d5e296640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d5812faf0 0x7f9d581311f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.302+0000 7f9d5e296640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58137ad0 con 0x7f9d5812faf0 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.302+0000 7f9d5e296640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f9d58137950 con 0x7f9d5809af50 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.302+0000 7f9d5e296640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f9d58137c50 con 0x7f9d5809a630 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d577fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809af50 0x7f9d58130cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d57fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5809a630 0x7f9d58130770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d57fff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5809a630 0x7f9d58130770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:42224/0 (socket says 172.21.15.47:42224) 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d57fff640 1 -- 172.21.15.47:0/2852298414 learned_addr learned my addr 172.21.15.47:0/2852298414 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d5c833640 1 --2- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d5812faf0 0x7f9d581311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d57fff640 1 -- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809af50 msgr2=0x7f9d58130cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d57fff640 1 --2- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809af50 0x7f9d58130cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d57fff640 1 -- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d5812faf0 msgr2=0x7f9d581311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d57fff640 1 --2- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d5812faf0 0x7f9d581311f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d57fff640 1 -- 172.21.15.47:0/2852298414 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d58133320 con 0x7f9d5809a630 2024-09-15T21:56:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d5c833640 1 --2- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d5812faf0 0x7f9d581311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d577fe640 1 --2- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809af50 0x7f9d58130cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.303+0000 7f9d57fff640 1 --2- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5809a630 0x7f9d58130770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800e9b0 tx=0x7f9d4800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.304+0000 7f9d54ff9640 1 -- 172.21.15.47:0/2852298414 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48017070 con 0x7f9d5809a630 2024-09-15T21:56:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.304+0000 7f9d54ff9640 1 -- 172.21.15.47:0/2852298414 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d4800c9a0 con 0x7f9d5809a630 2024-09-15T21:56:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.304+0000 7f9d5e296640 1 -- 172.21.15.47:0/2852298414 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d581321d0 con 0x7f9d5809a630 2024-09-15T21:56:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.304+0000 7f9d54ff9640 1 -- 172.21.15.47:0/2852298414 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800cd90 con 0x7f9d5809a630 2024-09-15T21:56:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.304+0000 7f9d5e296640 1 -- 172.21.15.47:0/2852298414 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d581319b0 con 0x7f9d5809a630 2024-09-15T21:56:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.305+0000 7f9d54ff9640 1 -- 172.21.15.47:0/2852298414 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f9d4801a070 con 0x7f9d5809a630 2024-09-15T21:56:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.305+0000 7f9d5e296640 1 -- 172.21.15.47:0/2852298414 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5809aa10 con 0x7f9d5809a630 2024-09-15T21:56:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.305+0000 7f9d54ff9640 1 -- 172.21.15.47:0/2852298414 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f9d4805da10 con 0x7f9d5809a630 2024-09-15T21:56:57.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.313+0000 7f9d54ff9640 1 -- 172.21.15.47:0/2852298414 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f9d4803cb60 con 0x7f9d5809a630 2024-09-15T21:56:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.426+0000 7f4a09e55640 1 -- 172.21.15.47:0/2781669495 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4a04130010 con 0x7f4a04134c60 2024-09-15T21:56:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.426+0000 7f49ebfff640 1 -- 172.21.15.47:0/2781669495 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49fc010040 con 0x7f4a04134c60 2024-09-15T21:56:57.429 INFO:teuthology.orchestra.run.smithi047.stdout:115964116994 2024-09-15T21:56:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.427+0000 7f47b02d7640 1 -- 172.21.15.47:0/934229667 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f47a812e220 con 0x7f47a8097030 2024-09-15T21:56:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.427+0000 7f479effd640 1 -- 172.21.15.47:0/934229667 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f47a404c030 con 0x7f47a8097030 2024-09-15T21:56:57.430 INFO:teuthology.orchestra.run.smithi047.stdout:115964116994 2024-09-15T21:56:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.428+0000 7fce90c08640 1 -- 172.21.15.47:0/4036288287 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fce8c075730 con 0x7fce8c12ab10 2024-09-15T21:56:57.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.428+0000 7f4a09e55640 1 -- 172.21.15.47:0/2781669495 >> v1:172.21.15.47:6800/1790418198 conn(0x7f49f0048000 legacy=0x7f49f004a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.428+0000 7f4a09e55640 1 -- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a04134c60 msgr2=0x7f4a041279f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.428+0000 7f4a09e55640 1 --2- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a04134c60 0x7f4a041279f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00ef90 tx=0x7f49fc00c490 comp rx=0 tx=0).stop 2024-09-15T21:56:57.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.429+0000 7fce7affd640 1 -- 172.21.15.47:0/4036288287 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fce8c073eb0 con 0x7fce8c12ab10 2024-09-15T21:56:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.429+0000 7f4a09e55640 1 -- 172.21.15.47:0/2781669495 shutdown_connections 2024-09-15T21:56:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.429+0000 7f4a09e55640 1 --2- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f4a04134c60 0x7f4a041279f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.429+0000 7f4a09e55640 1 --2- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f4a0409b120 0x7f4a041274b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.429+0000 7f4a09e55640 1 --2- 172.21.15.47:0/2781669495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a0409a630 0x7f4a04126f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.429+0000 7f4a09e55640 1 -- 172.21.15.47:0/2781669495 >> 172.21.15.47:0/2781669495 conn(0x7f4a0409eb90 msgr2=0x7f4a0412d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:57.432 INFO:teuthology.orchestra.run.smithi047.stdout:107374182402 2024-09-15T21:56:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.429+0000 7f4a09e55640 1 -- 172.21.15.47:0/2781669495 shutdown_connections 2024-09-15T21:56:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.429+0000 7f4a09e55640 1 -- 172.21.15.47:0/2781669495 wait complete. 2024-09-15T21:56:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.430+0000 7f47b02d7640 1 -- 172.21.15.47:0/934229667 >> v1:172.21.15.47:6800/1790418198 conn(0x7f478c047fd0 legacy=0x7f478c04a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.430+0000 7f47b02d7640 1 -- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a8097030 msgr2=0x7f47a812d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.430+0000 7f47b02d7640 1 --2- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a8097030 0x7f47a812d680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f47a40098c0 tx=0x7f47a40046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.430+0000 7f47b02d7640 1 -- 172.21.15.47:0/934229667 shutdown_connections 2024-09-15T21:56:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.430+0000 7f47b02d7640 1 --2- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f47a812f090 0x7f47a806e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.430+0000 7f47b02d7640 1 --2- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47a812e770 0x7f47a812dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.430+0000 7f47b02d7640 1 --2- 172.21.15.47:0/934229667 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f47a8097030 0x7f47a812d680 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.430+0000 7f47b02d7640 1 -- 172.21.15.47:0/934229667 >> 172.21.15.47:0/934229667 conn(0x7f47a809c5d0 msgr2=0x7f47a80996e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.430+0000 7f47b02d7640 1 -- 172.21.15.47:0/934229667 shutdown_connections 2024-09-15T21:56:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.430+0000 7f47b02d7640 1 -- 172.21.15.47:0/934229667 wait complete. 2024-09-15T21:56:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.431+0000 7fce90c08640 1 -- 172.21.15.47:0/4036288287 >> v1:172.21.15.47:6800/1790418198 conn(0x7fce6c0483d0 legacy=0x7fce6c04a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.431+0000 7fce90c08640 1 -- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12ab10 msgr2=0x7fce8c126680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.431+0000 7fce90c08640 1 --2- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12ab10 0x7fce8c126680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce80006e00 tx=0x7fce8000ee10 comp rx=0 tx=0).stop 2024-09-15T21:56:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.431+0000 7fce90c08640 1 -- 172.21.15.47:0/4036288287 shutdown_connections 2024-09-15T21:56:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.431+0000 7fce90c08640 1 --2- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fce8c12d8b0 0x7fce8c126bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.431+0000 7fce90c08640 1 --2- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fce8c12ab10 0x7fce8c126680 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.431+0000 7fce90c08640 1 --2- 172.21.15.47:0/4036288287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce8c12a1f0 0x7fce8c126140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.431+0000 7fce90c08640 1 -- 172.21.15.47:0/4036288287 >> 172.21.15.47:0/4036288287 conn(0x7fce8c09db80 msgr2=0x7fce8c09aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.431+0000 7fce90c08640 1 -- 172.21.15.47:0/4036288287 shutdown_connections 2024-09-15T21:56:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.431+0000 7fce90c08640 1 -- 172.21.15.47:0/4036288287 wait complete. 2024-09-15T21:56:57.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116994 for osd.2 2024-09-15T21:56:57.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116994 for osd.3 2024-09-15T21:56:57.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182404 got 107374182402 for osd.1 2024-09-15T21:56:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.465+0000 7f9d5e296640 1 -- 172.21.15.47:0/2852298414 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9d58133eb0 con 0x7f9d5809a630 2024-09-15T21:56:57.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.465+0000 7f9d54ff9640 1 -- 172.21.15.47:0/2852298414 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9d4803cdc0 con 0x7f9d5809a630 2024-09-15T21:56:57.467 INFO:teuthology.orchestra.run.smithi047.stdout:107374182402 2024-09-15T21:56:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.467+0000 7f9d5e296640 1 -- 172.21.15.47:0/2852298414 >> v1:172.21.15.47:6800/1790418198 conn(0x7f9d44047fd0 legacy=0x7f9d4404a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.467+0000 7f9d5e296640 1 -- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5809a630 msgr2=0x7f9d58130770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.467+0000 7f9d5e296640 1 --2- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5809a630 0x7f9d58130770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800e9b0 tx=0x7f9d4800ee70 comp rx=0 tx=0).stop 2024-09-15T21:56:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.467+0000 7f9d5e296640 1 -- 172.21.15.47:0/2852298414 shutdown_connections 2024-09-15T21:56:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.467+0000 7f9d5e296640 1 --2- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d5812faf0 0x7f9d581311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.467+0000 7f9d5e296640 1 --2- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9d5809af50 0x7f9d58130cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.467+0000 7f9d5e296640 1 --2- 172.21.15.47:0/2852298414 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9d5809a630 0x7f9d58130770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.467+0000 7f9d5e296640 1 -- 172.21.15.47:0/2852298414 >> 172.21.15.47:0/2852298414 conn(0x7f9d5809eb90 msgr2=0x7f9d5812f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.467+0000 7f9d5e296640 1 -- 172.21.15.47:0/2852298414 shutdown_connections 2024-09-15T21:56:57.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:57.467+0000 7f9d5e296640 1 -- 172.21.15.47:0/2852298414 wait complete. 2024-09-15T21:56:57.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182404 got 107374182402 for osd.0 2024-09-15T21:56:58.446 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:56:58.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:56:58.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:56:58.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:56:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fd00a0cb640 1 Processor -- start 2024-09-15T21:56:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fd62b3c2640 1 Processor -- start 2024-09-15T21:56:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fb36eae7640 1 Processor -- start 2024-09-15T21:56:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fd00a0cb640 1 -- start start 2024-09-15T21:56:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fb36eae7640 1 -- start start 2024-09-15T21:56:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fd62b3c2640 1 -- start start 2024-09-15T21:56:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fb36eae7640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb36812f780 0x7fb36812fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fb36eae7640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3681300a0 0x7fb368095c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fb36eae7640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb368096fb0 0x7fb368096150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fb36eae7640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb36806bdd0 con 0x7fb3681300a0 2024-09-15T21:56:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fb36eae7640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fb3680687d0 con 0x7fb36812f780 2024-09-15T21:56:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fb36eae7640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fb368068410 con 0x7fb368096fb0 2024-09-15T21:56:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fd00a0cb640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd004097180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.565+0000 7fd00a0cb640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd0040976c0 0x7fd004097b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd002ffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd0040976c0 0x7fd004097b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd0037fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd004097180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd002ffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd0040976c0 0x7fd004097b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:42238/0 (socket says 172.21.15.47:42238) 2024-09-15T21:56:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd002ffd640 1 -- 172.21.15.47:0/28141563 learned_addr learned my addr 172.21.15.47:0/28141563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd62b3c2640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd62412a1f0 0x7fd62412a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fb367fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3681300a0 0x7fb368095c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fb36d084640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb368096fb0 0x7fb368096150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fb36d084640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb368096fb0 0x7fb368096150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:42244/0 (socket says 172.21.15.47:42244) 2024-09-15T21:56:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fb36d084640 1 -- 172.21.15.47:0/4062410465 learned_addr learned my addr 172.21.15.47:0/4062410465 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:58.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fb36c883640 1 --2- 172.21.15.47:0/4062410465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb36812f780 0x7fb36812fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd62b3c2640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd62412ab10 0x7fd62412d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd62b3c2640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6240995c0 con 0x7fd62412d8b0 2024-09-15T21:56:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd62b3c2640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd62409f9d0 con 0x7fd62412a1f0 2024-09-15T21:56:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd62b3c2640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd624099990 con 0x7fd62412ab10 2024-09-15T21:56:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd00a0cb640 1 --2- 172.21.15.47:0/28141563 >> v2:172.21.15.47:3300/0 conn(0x7fd004133da0 0x7fd004136190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd00a0cb640 1 -- 172.21.15.47:0/28141563 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd00406c060 con 0x7fd004133da0 2024-09-15T21:56:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd00a0cb640 1 -- 172.21.15.47:0/28141563 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd004096d80 con 0x7fd0040983c0 2024-09-15T21:56:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd00a0cb640 1 -- 172.21.15.47:0/28141563 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd0040991b0 con 0x7fd0040976c0 2024-09-15T21:56:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fb36d084640 1 -- 172.21.15.47:0/4062410465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb36812f780 msgr2=0x7fb36812fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd003fff640 1 --2- 172.21.15.47:0/28141563 >> v2:172.21.15.47:3300/0 conn(0x7fd004133da0 0x7fd004136190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd002ffd640 1 -- 172.21.15.47:0/28141563 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 msgr2=0x7fd004097180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd002ffd640 1 --2- 172.21.15.47:0/28141563 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd004097180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd002ffd640 1 -- 172.21.15.47:0/28141563 >> v2:172.21.15.47:3300/0 conn(0x7fd004133da0 msgr2=0x7fd004136190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd002ffd640 1 --2- 172.21.15.47:0/28141563 >> v2:172.21.15.47:3300/0 conn(0x7fd004133da0 0x7fd004136190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd002ffd640 1 -- 172.21.15.47:0/28141563 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0041366d0 con 0x7fd0040976c0 2024-09-15T21:56:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd0037fe640 1 --2- 172.21.15.47:0/28141563 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd004097180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd62915e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd62412a1f0 0x7fd62412a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd62915e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd62412a1f0 0x7fd62412a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46064/0 (socket says 172.21.15.47:46064) 2024-09-15T21:56:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd62915e640 1 -- 172.21.15.47:0/3268740389 learned_addr learned my addr 172.21.15.47:0/3268740389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fd62895d640 1 --2- 172.21.15.47:0/3268740389 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd62412ab10 0x7fd62412d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fb36d084640 1 --2- 172.21.15.47:0/4062410465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb36812f780 0x7fb36812fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fb36d084640 1 -- 172.21.15.47:0/4062410465 >> v2:172.21.15.47:3300/0 conn(0x7fb3681300a0 msgr2=0x7fb368095c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fb36d084640 1 --2- 172.21.15.47:0/4062410465 >> v2:172.21.15.47:3300/0 conn(0x7fb3681300a0 0x7fb368095c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.566+0000 7fb36d084640 1 -- 172.21.15.47:0/4062410465 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb368096690 con 0x7fb368096fb0 2024-09-15T21:56:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.567+0000 7fd62915e640 1 -- 172.21.15.47:0/3268740389 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd62412ab10 msgr2=0x7fd62412d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.567+0000 7fd62915e640 1 --2- 172.21.15.47:0/3268740389 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd62412ab10 0x7fd62412d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.567+0000 7fd62915e640 1 -- 172.21.15.47:0/3268740389 >> v1:172.21.15.47:6789/0 conn(0x7fd62412d8b0 legacy=0x7fd6241346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.567+0000 7fd62915e640 1 -- 172.21.15.47:0/3268740389 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd624099ec0 con 0x7fd62412a1f0 2024-09-15T21:56:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.567+0000 7fb36c883640 1 --2- 172.21.15.47:0/4062410465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb36812f780 0x7fb36812fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.567+0000 7fd002ffd640 1 --2- 172.21.15.47:0/28141563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd0040976c0 0x7fd004097b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009870 tx=0x7fcff802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b540f901a436860b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fb36d084640 1 --2- 172.21.15.47:0/4062410465 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb368096fb0 0x7fb368096150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb360009e80 tx=0x7fb36002a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4169c1af06afd348 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fd62915e640 1 --2- 172.21.15.47:0/3268740389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62412a1f0 0x7fd62412a5d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd6200098a0 tx=0x7fd62002a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e874e1f0816b2199 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fd0027fc640 1 -- 172.21.15.47:0/28141563 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8037070 con 0x7fd0040976c0 2024-09-15T21:56:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fd0027fc640 1 -- 172.21.15.47:0/28141563 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff8032440 con 0x7fd0040976c0 2024-09-15T21:56:58.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fd0027fc640 1 -- 172.21.15.47:0/28141563 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8032810 con 0x7fd0040976c0 2024-09-15T21:56:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fb3677fe640 1 -- 172.21.15.47:0/4062410465 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360036070 con 0x7fb368096fb0 2024-09-15T21:56:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fb3677fe640 1 -- 172.21.15.47:0/4062410465 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb360031600 con 0x7fb368096fb0 2024-09-15T21:56:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fb3677fe640 1 -- 172.21.15.47:0/4062410465 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600319d0 con 0x7fb368096fb0 2024-09-15T21:56:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fd61bfff640 1 -- 172.21.15.47:0/3268740389 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620037070 con 0x7fd62412a1f0 2024-09-15T21:56:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fd61bfff640 1 -- 172.21.15.47:0/3268740389 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd620032440 con 0x7fd62412a1f0 2024-09-15T21:56:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.568+0000 7fd61bfff640 1 -- 172.21.15.47:0/3268740389 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620032810 con 0x7fd62412a1f0 2024-09-15T21:56:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd00a0cb640 1 -- 172.21.15.47:0/28141563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd0040976c0 msgr2=0x7fd004097b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd00a0cb640 1 --2- 172.21.15.47:0/28141563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd0040976c0 0x7fd004097b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009870 tx=0x7fcff802a130 comp rx=0 tx=0).stop 2024-09-15T21:56:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd00a0cb640 1 -- 172.21.15.47:0/28141563 shutdown_connections 2024-09-15T21:56:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd00a0cb640 1 --2- 172.21.15.47:0/28141563 >> v2:172.21.15.47:3300/0 conn(0x7fd004133da0 0x7fd004136190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd00a0cb640 1 --2- 172.21.15.47:0/28141563 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd0040976c0 0x7fd004097b20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd00a0cb640 1 --2- 172.21.15.47:0/28141563 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd004097180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd00a0cb640 1 -- 172.21.15.47:0/28141563 >> 172.21.15.47:0/28141563 conn(0x7fd00409fdf0 msgr2=0x7fd0040a0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fb36eae7640 1 -- 172.21.15.47:0/4062410465 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb368096fb0 msgr2=0x7fb368096150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fb36eae7640 1 --2- 172.21.15.47:0/4062410465 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb368096fb0 0x7fb368096150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb360009e80 tx=0x7fb36002a590 comp rx=0 tx=0).stop 2024-09-15T21:56:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd62b3c2640 1 -- 172.21.15.47:0/3268740389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62412a1f0 msgr2=0x7fd62412a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd62b3c2640 1 --2- 172.21.15.47:0/3268740389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62412a1f0 0x7fd62412a5d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd6200098a0 tx=0x7fd62002a1f0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd00a0cb640 1 -- 172.21.15.47:0/28141563 shutdown_connections 2024-09-15T21:56:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fb36eae7640 1 -- 172.21.15.47:0/4062410465 shutdown_connections 2024-09-15T21:56:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fb36eae7640 1 --2- 172.21.15.47:0/4062410465 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb368096fb0 0x7fb368096150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fb36eae7640 1 --2- 172.21.15.47:0/4062410465 >> v2:172.21.15.47:3300/0 conn(0x7fb3681300a0 0x7fb368095c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fb36eae7640 1 --2- 172.21.15.47:0/4062410465 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb36812f780 0x7fb36812fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fb36eae7640 1 -- 172.21.15.47:0/4062410465 >> 172.21.15.47:0/4062410465 conn(0x7fb36809c5d0 msgr2=0x7fb36809c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd00a0cb640 1 -- 172.21.15.47:0/28141563 wait complete. 2024-09-15T21:56:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd62b3c2640 1 -- 172.21.15.47:0/3268740389 shutdown_connections 2024-09-15T21:56:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd62b3c2640 1 --2- 172.21.15.47:0/3268740389 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd62412ab10 0x7fd62412d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd62b3c2640 1 --2- 172.21.15.47:0/3268740389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62412a1f0 0x7fd62412a5d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd62b3c2640 1 -- 172.21.15.47:0/3268740389 >> 172.21.15.47:0/3268740389 conn(0x7fd62409db80 msgr2=0x7fd62409df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fb36eae7640 1 -- 172.21.15.47:0/4062410465 shutdown_connections 2024-09-15T21:56:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fb36eae7640 1 -- 172.21.15.47:0/4062410465 wait complete. 2024-09-15T21:56:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd62b3c2640 1 -- 172.21.15.47:0/3268740389 shutdown_connections 2024-09-15T21:56:58.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.569+0000 7fd62b3c2640 1 -- 172.21.15.47:0/3268740389 wait complete. 2024-09-15T21:56:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.570+0000 7fd00a0cb640 1 Processor -- start 2024-09-15T21:56:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.570+0000 7fb36eae7640 1 Processor -- start 2024-09-15T21:56:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.570+0000 7fb36eae7640 1 -- start start 2024-09-15T21:56:58.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.570+0000 7fd62b3c2640 1 Processor -- start 2024-09-15T21:56:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.570+0000 7fd00a0cb640 1 -- start start 2024-09-15T21:56:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.570+0000 7fd62b3c2640 1 -- start start 2024-09-15T21:56:58.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd00a0cb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0040976c0 0x7fd00406de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd00a0cb640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd00406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd00a0cb640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd004133da0 0x7fd00406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd00a0cb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd004136c30 con 0x7fd0040976c0 2024-09-15T21:56:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd00a0cb640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd004136ab0 con 0x7fd0040983c0 2024-09-15T21:56:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd00a0cb640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd004136db0 con 0x7fd004133da0 2024-09-15T21:56:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fb36eae7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb368096fb0 0x7fb36812b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fb36eae7640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb36812f780 0x7fb36806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fb36eae7640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3681300a0 0x7fb36806e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fb36eae7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb368136900 con 0x7fb368096fb0 2024-09-15T21:56:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fb36eae7640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fb368096a60 con 0x7fb3681300a0 2024-09-15T21:56:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fb36eae7640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fb368136a80 con 0x7fb36812f780 2024-09-15T21:56:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62b3c2640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd62412a1f0 0x7fd62408cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62b3c2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62412ab10 0x7fd62408d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd002ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd00406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd002ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd00406e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44960/0 (socket says 172.21.15.47:44960) 2024-09-15T21:56:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd002ffd640 1 -- 172.21.15.47:0/2719736905 learned_addr learned my addr 172.21.15.47:0/2719736905 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd0037fe640 1 --2- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0040976c0 0x7fd00406de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd003fff640 1 --2- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd004133da0 0x7fd00406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62b3c2640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd62412d8b0 0x7fd62408da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62b3c2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd62412de80 con 0x7fd62412ab10 2024-09-15T21:56:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62b3c2640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fd62412dd00 con 0x7fd62412a1f0 2024-09-15T21:56:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62b3c2640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fd62412e000 con 0x7fd62412d8b0 2024-09-15T21:56:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62995f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd62412d8b0 0x7fd62408da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62995f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd62412d8b0 0x7fd62408da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:42266/0 (socket says 172.21.15.47:42266) 2024-09-15T21:56:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62995f640 1 -- 172.21.15.47:0/1005416232 learned_addr learned my addr 172.21.15.47:0/1005416232 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62915e640 1 --2- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd62412a1f0 0x7fd62408cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd62895d640 1 --2- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62412ab10 0x7fd62408d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd002ffd640 1 -- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd004133da0 msgr2=0x7fd00406e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd002ffd640 1 --2- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd004133da0 0x7fd00406e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd002ffd640 1 -- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0040976c0 msgr2=0x7fd00406de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd002ffd640 1 --2- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0040976c0 0x7fd00406de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fd002ffd640 1 -- 172.21.15.47:0/2719736905 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0040955e0 con 0x7fd0040983c0 2024-09-15T21:56:58.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fd0037fe640 1 --2- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0040976c0 0x7fd00406de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fb36d084640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3681300a0 0x7fb36806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fb36d084640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3681300a0 0x7fb36806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44970/0 (socket says 172.21.15.47:44970) 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fb36d084640 1 -- 172.21.15.47:0/2159429022 learned_addr learned my addr 172.21.15.47:0/2159429022 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.571+0000 7fb367fff640 1 --2- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb36812f780 0x7fb36806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fb36c883640 1 --2- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb368096fb0 0x7fb36812b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fb36d084640 1 -- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb36812f780 msgr2=0x7fb36806e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fb36d084640 1 --2- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb36812f780 0x7fb36806e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fb36d084640 1 -- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb368096fb0 msgr2=0x7fb36812b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fb36d084640 1 --2- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb368096fb0 0x7fb36812b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fb36d084640 1 -- 172.21.15.47:0/2159429022 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36808ff30 con 0x7fb3681300a0 2024-09-15T21:56:58.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fb36c883640 1 --2- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb368096fb0 0x7fb36812b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:56:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fd62895d640 1 -- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd62412d8b0 msgr2=0x7fd62408da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fd62895d640 1 --2- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd62412d8b0 0x7fd62408da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fd62895d640 1 -- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd62412a1f0 msgr2=0x7fd62408cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fd62895d640 1 --2- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd62412a1f0 0x7fd62408cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fd62895d640 1 -- 172.21.15.47:0/1005416232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd624129380 con 0x7fd62412ab10 2024-09-15T21:56:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fd62995f640 1 --2- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd62412d8b0 0x7fd62408da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fb36d084640 1 --2- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3681300a0 0x7fb36806e720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb360009e80 tx=0x7fb360004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fd002ffd640 1 --2- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd00406e380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcff802a170 tx=0x7fcff80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:58.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fd62895d640 1 --2- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62412ab10 0x7fd62408d520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e850 tx=0x7fd61400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fb3657fa640 1 -- 172.21.15.47:0/2159429022 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360041020 con 0x7fb3681300a0 2024-09-15T21:56:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fb3657fa640 1 -- 172.21.15.47:0/2159429022 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3600086b0 con 0x7fb3681300a0 2024-09-15T21:56:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.572+0000 7fcfebfff640 1 -- 172.21.15.47:0/2719736905 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8037040 con 0x7fd0040983c0 2024-09-15T21:56:58.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fd00a0cb640 1 -- 172.21.15.47:0/2719736905 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd004070120 con 0x7fd0040983c0 2024-09-15T21:56:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fd619ffb640 1 -- 172.21.15.47:0/1005416232 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614017070 con 0x7fd62412ab10 2024-09-15T21:56:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fcfebfff640 1 -- 172.21.15.47:0/2719736905 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff8042070 con 0x7fd0040983c0 2024-09-15T21:56:58.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fd619ffb640 1 -- 172.21.15.47:0/1005416232 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61400c940 con 0x7fd62412ab10 2024-09-15T21:56:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fcfebfff640 1 -- 172.21.15.47:0/2719736905 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8030880 con 0x7fd0040983c0 2024-09-15T21:56:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fd00a0cb640 1 -- 172.21.15.47:0/2719736905 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd004095d30 con 0x7fd0040983c0 2024-09-15T21:56:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fd619ffb640 1 -- 172.21.15.47:0/1005416232 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614017070 con 0x7fd62412ab10 2024-09-15T21:56:58.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fd62b3c2640 1 -- 172.21.15.47:0/1005416232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62408f000 con 0x7fd62412ab10 2024-09-15T21:56:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fb36eae7640 1 -- 172.21.15.47:0/2159429022 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb368098200 con 0x7fb3681300a0 2024-09-15T21:56:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fb36eae7640 1 -- 172.21.15.47:0/2159429022 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36806fad0 con 0x7fb3681300a0 2024-09-15T21:56:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.573+0000 7fb3657fa640 1 -- 172.21.15.47:0/2159429022 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360036040 con 0x7fb3681300a0 2024-09-15T21:56:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.574+0000 7fb36eae7640 1 -- 172.21.15.47:0/2159429022 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb36812fb60 con 0x7fb3681300a0 2024-09-15T21:56:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.574+0000 7fd00a0cb640 1 -- 172.21.15.47:0/2719736905 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd004097b20 con 0x7fd0040983c0 2024-09-15T21:56:58.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.574+0000 7fd62b3c2640 1 -- 172.21.15.47:0/1005416232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62408e720 con 0x7fd62412ab10 2024-09-15T21:56:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.574+0000 7fb3657fa640 1 -- 172.21.15.47:0/2159429022 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fb360044020 con 0x7fb3681300a0 2024-09-15T21:56:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.575+0000 7fb3657fa640 1 -- 172.21.15.47:0/2159429022 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fb36007e620 con 0x7fb3681300a0 2024-09-15T21:56:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.575+0000 7fd619ffb640 1 -- 172.21.15.47:0/1005416232 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fd61401a070 con 0x7fd62412ab10 2024-09-15T21:56:58.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.575+0000 7fd62b3c2640 1 -- 172.21.15.47:0/1005416232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd624075730 con 0x7fd62412ab10 2024-09-15T21:56:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.575+0000 7fcfebfff640 1 -- 172.21.15.47:0/2719736905 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fcff8030a20 con 0x7fd0040983c0 2024-09-15T21:56:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.575+0000 7fd619ffb640 1 -- 172.21.15.47:0/1005416232 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fd61405d660 con 0x7fd62412ab10 2024-09-15T21:56:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.576+0000 7fcfebfff640 1 -- 172.21.15.47:0/2719736905 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fcff807e430 con 0x7fd0040983c0 2024-09-15T21:56:58.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.579+0000 7fb3657fa640 1 -- 172.21.15.47:0/2159429022 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fb36004b070 con 0x7fb3681300a0 2024-09-15T21:56:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.580+0000 7fd619ffb640 1 -- 172.21.15.47:0/1005416232 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fd61403c730 con 0x7fd62412ab10 2024-09-15T21:56:58.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.585+0000 7fcfebfff640 1 -- 172.21.15.47:0/2719736905 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fcff807d4e0 con 0x7fd0040983c0 2024-09-15T21:56:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.618+0000 7f503b983640 1 Processor -- start 2024-09-15T21:56:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.618+0000 7f503b983640 1 -- start start 2024-09-15T21:56:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f503b983640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409a630 0x7f503409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f503b983640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f503409af50 0x7f503412f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f503971f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409a630 0x7f503409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f5038f1e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f503409af50 0x7f503412f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f503971f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409a630 0x7f503409aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:44986/0 (socket says 172.21.15.47:44986) 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f5038f1e640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f503409af50 0x7f503412f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46096/0 (socket says 172.21.15.47:46096) 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f503b983640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 0x7f503412fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f503971f640 1 -- 172.21.15.47:0/2719527479 learned_addr learned my addr 172.21.15.47:0/2719527479 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f503b983640 1 -- 172.21.15.47:0/2719527479 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f503409b350 con 0x7f503409af50 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f503b983640 1 -- 172.21.15.47:0/2719527479 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f503409b600 con 0x7f503409a630 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.619+0000 7f503b983640 1 -- 172.21.15.47:0/2719527479 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f503406d480 con 0x7f503412f920 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.620+0000 7f5039f20640 1 --2- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 0x7f503412fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.620+0000 7f5038f1e640 1 -- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 msgr2=0x7f503412fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.620+0000 7f5038f1e640 1 --2- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 0x7f503412fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.620+0000 7f5038f1e640 1 -- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409a630 msgr2=0x7f503409aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.620+0000 7f5038f1e640 1 --2- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409a630 0x7f503409aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.620+0000 7f5038f1e640 1 -- 172.21.15.47:0/2719527479 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503406d140 con 0x7f503409af50 2024-09-15T21:56:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.621+0000 7f503971f640 1 --2- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409a630 0x7f503409aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.621+0000 7f5038f1e640 1 --2- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409af50 0x7f503412f3e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f50240097a0 tx=0x7f5024029fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38a9fbf35d4a5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.621+0000 7f502bfff640 1 -- 172.21.15.47:0/2719527479 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024037070 con 0x7f503409af50 2024-09-15T21:56:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.621+0000 7f502bfff640 1 -- 172.21.15.47:0/2719527479 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f502402ad20 con 0x7f503409af50 2024-09-15T21:56:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.621+0000 7f502bfff640 1 -- 172.21.15.47:0/2719527479 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024032660 con 0x7f503409af50 2024-09-15T21:56:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.622+0000 7f503b983640 1 -- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409af50 msgr2=0x7f503412f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.622+0000 7f503b983640 1 --2- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409af50 0x7f503412f3e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f50240097a0 tx=0x7f5024029fd0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.622+0000 7f503b983640 1 -- 172.21.15.47:0/2719527479 shutdown_connections 2024-09-15T21:56:58.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.622+0000 7f503b983640 1 --2- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 0x7f503412fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.622+0000 7f503b983640 1 --2- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409af50 0x7f503412f3e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.622+0000 7f503b983640 1 --2- 172.21.15.47:0/2719527479 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409a630 0x7f503409aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.622+0000 7f503b983640 1 -- 172.21.15.47:0/2719527479 >> 172.21.15.47:0/2719527479 conn(0x7f503409eb90 msgr2=0x7f503409efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:58.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.622+0000 7f503b983640 1 -- 172.21.15.47:0/2719527479 shutdown_connections 2024-09-15T21:56:58.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.622+0000 7f503b983640 1 -- 172.21.15.47:0/2719527479 wait complete. 2024-09-15T21:56:58.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.624+0000 7f503b983640 1 Processor -- start 2024-09-15T21:56:58.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.624+0000 7f503b983640 1 -- start start 2024-09-15T21:56:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f503b983640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409a630 0x7f5034130d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f503b983640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409af50 0x7f5034131250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f503b983640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 0x7f5034131790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f503b983640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f50341368b0 con 0x7f503409a630 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f503b983640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f5034136730 con 0x7f503409af50 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f503b983640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f5034136a30 con 0x7f503412f920 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f503971f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409a630 0x7f5034130d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f503971f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409a630 0x7f5034130d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46108/0 (socket says 172.21.15.47:46108) 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f503971f640 1 -- 172.21.15.47:0/3123320675 learned_addr learned my addr 172.21.15.47:0/3123320675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f5039f20640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 0x7f5034131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.625+0000 7f5038f1e640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409af50 0x7f5034131250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f5039f20640 1 -- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409af50 msgr2=0x7f5034131250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f5039f20640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409af50 0x7f5034131250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f5039f20640 1 -- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409a630 msgr2=0x7f5034130d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f5039f20640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409a630 0x7f5034130d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f5039f20640 1 -- 172.21.15.47:0/3123320675 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5034132880 con 0x7f503412f920 2024-09-15T21:56:58.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f503971f640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409a630 0x7f5034130d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f5038f1e640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409af50 0x7f5034131250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f5039f20640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 0x7f5034131790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f502c009d90 tx=0x7f502c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:58.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f5029ffb640 1 -- 172.21.15.47:0/3123320675 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c018020 con 0x7f503412f920 2024-09-15T21:56:58.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f5029ffb640 1 -- 172.21.15.47:0/3123320675 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f502c014480 con 0x7f503412f920 2024-09-15T21:56:58.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f503b983640 1 -- 172.21.15.47:0/3123320675 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5034133a80 con 0x7f503412f920 2024-09-15T21:56:58.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.626+0000 7f5029ffb640 1 -- 172.21.15.47:0/3123320675 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c014870 con 0x7f503412f920 2024-09-15T21:56:58.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.627+0000 7f503b983640 1 -- 172.21.15.47:0/3123320675 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f50341349b0 con 0x7f503412f920 2024-09-15T21:56:58.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.628+0000 7f5029ffb640 1 -- 172.21.15.47:0/3123320675 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f502c005d10 con 0x7f503412f920 2024-09-15T21:56:58.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.628+0000 7f503b983640 1 -- 172.21.15.47:0/3123320675 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50341345f0 con 0x7f503412f920 2024-09-15T21:56:58.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.631+0000 7f5029ffb640 1 -- 172.21.15.47:0/3123320675 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f502c05e700 con 0x7f503412f920 2024-09-15T21:56:58.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.636+0000 7f5029ffb640 1 -- 172.21.15.47:0/3123320675 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f50341345f0 con 0x7f503412f920 2024-09-15T21:56:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.732+0000 7fd62b3c2640 1 -- 172.21.15.47:0/1005416232 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd62412a5d0 con 0x7fd62412ab10 2024-09-15T21:56:58.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.732+0000 7fd619ffb640 1 -- 172.21.15.47:0/1005416232 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd61403c990 con 0x7fd62412ab10 2024-09-15T21:56:58.735 INFO:teuthology.orchestra.run.smithi047.stdout:115964116995 2024-09-15T21:56:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.735+0000 7fd62b3c2640 1 -- 172.21.15.47:0/1005416232 >> v1:172.21.15.47:6800/1790418198 conn(0x7fd6080481d0 legacy=0x7fd60804a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.735+0000 7fd62b3c2640 1 -- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62412ab10 msgr2=0x7fd62408d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.735+0000 7fd62b3c2640 1 --2- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62412ab10 0x7fd62408d520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e850 tx=0x7fd61400ed10 comp rx=0 tx=0).stop 2024-09-15T21:56:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.735+0000 7fd62b3c2640 1 -- 172.21.15.47:0/1005416232 shutdown_connections 2024-09-15T21:56:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.735+0000 7fd62b3c2640 1 --2- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd62412d8b0 0x7fd62408da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.735+0000 7fd62b3c2640 1 --2- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd62412ab10 0x7fd62408d520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.735+0000 7fd62b3c2640 1 --2- 172.21.15.47:0/1005416232 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd62412a1f0 0x7fd62408cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.735+0000 7fd62b3c2640 1 -- 172.21.15.47:0/1005416232 >> 172.21.15.47:0/1005416232 conn(0x7fd62409db80 msgr2=0x7fd62409a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.735+0000 7fd62b3c2640 1 -- 172.21.15.47:0/1005416232 shutdown_connections 2024-09-15T21:56:58.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.735+0000 7fd62b3c2640 1 -- 172.21.15.47:0/1005416232 wait complete. 2024-09-15T21:56:58.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.744+0000 7fd00a0cb640 1 -- 172.21.15.47:0/2719736905 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd004075b40 con 0x7fd0040983c0 2024-09-15T21:56:58.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.745+0000 7fcfebfff640 1 -- 172.21.15.47:0/2719736905 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcff8030d10 con 0x7fd0040983c0 2024-09-15T21:56:58.747 INFO:teuthology.orchestra.run.smithi047.stdout:115964116995 2024-09-15T21:56:58.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116995 for osd.3 2024-09-15T21:56:58.750 DEBUG:teuthology.parallel:result is None 2024-09-15T21:56:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.747+0000 7fd00a0cb640 1 -- 172.21.15.47:0/2719736905 >> v1:172.21.15.47:6800/1790418198 conn(0x7fcff0047fd0 legacy=0x7fcff004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.747+0000 7fd00a0cb640 1 -- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 msgr2=0x7fd00406e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.747+0000 7fd00a0cb640 1 --2- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd00406e380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcff802a170 tx=0x7fcff80046c0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.747+0000 7fd00a0cb640 1 -- 172.21.15.47:0/2719736905 shutdown_connections 2024-09-15T21:56:58.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.747+0000 7fd00a0cb640 1 --2- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fd004133da0 0x7fd00406e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.747+0000 7fd00a0cb640 1 --2- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fd0040983c0 0x7fd00406e380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.747+0000 7fd00a0cb640 1 --2- 172.21.15.47:0/2719736905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd0040976c0 0x7fd00406de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.747+0000 7fd00a0cb640 1 -- 172.21.15.47:0/2719736905 >> 172.21.15.47:0/2719736905 conn(0x7fd00409fdf0 msgr2=0x7fd00412ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.747+0000 7fd00a0cb640 1 -- 172.21.15.47:0/2719736905 shutdown_connections 2024-09-15T21:56:58.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.747+0000 7fd00a0cb640 1 -- 172.21.15.47:0/2719736905 wait complete. 2024-09-15T21:56:58.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116995 for osd.2 2024-09-15T21:56:58.762 DEBUG:teuthology.parallel:result is None 2024-09-15T21:56:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.760+0000 7fb36eae7640 1 -- 172.21.15.47:0/2159429022 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb36812e2a0 con 0x7fb3681300a0 2024-09-15T21:56:58.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.760+0000 7fb3657fa640 1 -- 172.21.15.47:0/2159429022 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb36007d630 con 0x7fb3681300a0 2024-09-15T21:56:58.763 INFO:teuthology.orchestra.run.smithi047.stdout:107374182404 2024-09-15T21:56:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.762+0000 7fb36eae7640 1 -- 172.21.15.47:0/2159429022 >> v1:172.21.15.47:6800/1790418198 conn(0x7fb34c047fd0 legacy=0x7fb34c04a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.762+0000 7fb36eae7640 1 -- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3681300a0 msgr2=0x7fb36806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.762+0000 7fb36eae7640 1 --2- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3681300a0 0x7fb36806e720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb360009e80 tx=0x7fb360004a40 comp rx=0 tx=0).stop 2024-09-15T21:56:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.762+0000 7fb36eae7640 1 -- 172.21.15.47:0/2159429022 shutdown_connections 2024-09-15T21:56:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.762+0000 7fb36eae7640 1 --2- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fb3681300a0 0x7fb36806e720 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.762+0000 7fb36eae7640 1 --2- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fb36812f780 0x7fb36806e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.762+0000 7fb36eae7640 1 --2- 172.21.15.47:0/2159429022 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb368096fb0 0x7fb36812b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.762+0000 7fb36eae7640 1 -- 172.21.15.47:0/2159429022 >> 172.21.15.47:0/2159429022 conn(0x7fb36809c5d0 msgr2=0x7fb368099770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.762+0000 7fb36eae7640 1 -- 172.21.15.47:0/2159429022 shutdown_connections 2024-09-15T21:56:58.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.762+0000 7fb36eae7640 1 -- 172.21.15.47:0/2159429022 wait complete. 2024-09-15T21:56:58.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182404 got 107374182404 for osd.1 2024-09-15T21:56:58.776 DEBUG:teuthology.parallel:result is None 2024-09-15T21:56:58.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.800+0000 7f503b983640 1 -- 172.21.15.47:0/3123320675 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f503412f3e0 con 0x7f503412f920 2024-09-15T21:56:58.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.800+0000 7f5029ffb640 1 -- 172.21.15.47:0/3123320675 <== mon.2 v2:172.21.15.47:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f502c022030 con 0x7f503412f920 2024-09-15T21:56:58.802 INFO:teuthology.orchestra.run.smithi047.stdout:107374182404 2024-09-15T21:56:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.802+0000 7f503b983640 1 -- 172.21.15.47:0/3123320675 >> v1:172.21.15.47:6800/1790418198 conn(0x7f5018048000 legacy=0x7f501804a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.802+0000 7f503b983640 1 -- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 msgr2=0x7f5034131790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.802+0000 7f503b983640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 0x7f5034131790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f502c009d90 tx=0x7f502c00c490 comp rx=0 tx=0).stop 2024-09-15T21:56:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.802+0000 7f503b983640 1 -- 172.21.15.47:0/3123320675 shutdown_connections 2024-09-15T21:56:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.802+0000 7f503b983640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f503412f920 0x7f5034131790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.802+0000 7f503b983640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f503409af50 0x7f5034131250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.802+0000 7f503b983640 1 --2- 172.21.15.47:0/3123320675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f503409a630 0x7f5034130d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.802+0000 7f503b983640 1 -- 172.21.15.47:0/3123320675 >> 172.21.15.47:0/3123320675 conn(0x7f503409eb90 msgr2=0x7f503409df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:58.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.802+0000 7f503b983640 1 -- 172.21.15.47:0/3123320675 shutdown_connections 2024-09-15T21:56:58.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.802+0000 7f503b983640 1 -- 172.21.15.47:0/3123320675 wait complete. 2024-09-15T21:56:58.816 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182404 got 107374182404 for osd.0 2024-09-15T21:56:58.816 DEBUG:teuthology.parallel:result is None 2024-09-15T21:56:58.816 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:56:58.816 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:56:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d444b1640 1 Processor -- start 2024-09-15T21:56:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d444b1640 1 -- start start 2024-09-15T21:56:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d444b1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d3c06dcd0 0x7f1d3c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d444b1640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06e5f0 0x7f1d3c06e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d444b1640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c136100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d444b1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1d3c073d30 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d444b1640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c070610 con 0x7f1d3c133d10 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d444b1640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f1d3c06f970 con 0x7f1d3c06e5f0 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d4224d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d3c06dcd0 0x7f1d3c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d42a4e640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c136100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d41a4c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06e5f0 0x7f1d3c06e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d4224d640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d3c06dcd0 0x7f1d3c06e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46110/0 (socket says 172.21.15.47:46110) 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d4224d640 1 -- 172.21.15.47:0/3503684781 learned_addr learned my addr 172.21.15.47:0/3503684781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.955+0000 7f1d42a4e640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c136100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45002/0 (socket says 172.21.15.47:45002) 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.956+0000 7f1d42a4e640 1 -- 172.21.15.47:0/3503684781 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06e5f0 msgr2=0x7f1d3c06e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.956+0000 7f1d42a4e640 1 --2- 172.21.15.47:0/3503684781 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06e5f0 0x7f1d3c06e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.956+0000 7f1d42a4e640 1 -- 172.21.15.47:0/3503684781 >> v2:172.21.15.47:3300/0 conn(0x7f1d3c06dcd0 msgr2=0x7f1d3c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.956+0000 7f1d42a4e640 1 --2- 172.21.15.47:0/3503684781 >> v2:172.21.15.47:3300/0 conn(0x7f1d3c06dcd0 0x7f1d3c06e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.956+0000 7f1d42a4e640 1 -- 172.21.15.47:0/3503684781 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c06ef10 con 0x7f1d3c133d10 2024-09-15T21:56:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.956+0000 7f1d4224d640 1 --2- 172.21.15.47:0/3503684781 >> v2:172.21.15.47:3300/0 conn(0x7f1d3c06dcd0 0x7f1d3c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.956+0000 7f1d41a4c640 1 --2- 172.21.15.47:0/3503684781 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06e5f0 0x7f1d3c06e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.956+0000 7f1d42a4e640 1 --2- 172.21.15.47:0/3503684781 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c136100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009d90 tx=0x7f1d3402a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b215be069a9d478 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d4124b640 1 -- 172.21.15.47:0/3503684781 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34007740 con 0x7f1d3c133d10 2024-09-15T21:56:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d4124b640 1 -- 172.21.15.47:0/3503684781 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d340078e0 con 0x7f1d3c133d10 2024-09-15T21:56:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d4124b640 1 -- 172.21.15.47:0/3503684781 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34007cb0 con 0x7f1d3c133d10 2024-09-15T21:56:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d444b1640 1 -- 172.21.15.47:0/3503684781 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 msgr2=0x7f1d3c136100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d444b1640 1 --2- 172.21.15.47:0/3503684781 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c136100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009d90 tx=0x7f1d3402a490 comp rx=0 tx=0).stop 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d444b1640 1 -- 172.21.15.47:0/3503684781 shutdown_connections 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d444b1640 1 --2- 172.21.15.47:0/3503684781 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c136100 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d444b1640 1 --2- 172.21.15.47:0/3503684781 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06e5f0 0x7f1d3c06e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d444b1640 1 --2- 172.21.15.47:0/3503684781 >> v2:172.21.15.47:3300/0 conn(0x7f1d3c06dcd0 0x7f1d3c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d444b1640 1 -- 172.21.15.47:0/3503684781 >> 172.21.15.47:0/3503684781 conn(0x7f1d3c09ed50 msgr2=0x7f1d3c09f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d444b1640 1 -- 172.21.15.47:0/3503684781 shutdown_connections 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.957+0000 7f1d444b1640 1 -- 172.21.15.47:0/3503684781 wait complete. 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.958+0000 7f1d444b1640 1 Processor -- start 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.958+0000 7f1d444b1640 1 -- start start 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d444b1640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06dcd0 0x7f1d3c1330c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d444b1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c06e5f0 0x7f1d3c133600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d444b1640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c094940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d444b1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c08d370 con 0x7f1d3c06e5f0 2024-09-15T21:56:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d444b1640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c08d1f0 con 0x7f1d3c133d10 2024-09-15T21:56:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d444b1640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f1d3c08d4f0 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d4224d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06dcd0 0x7f1d3c1330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d4224d640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06dcd0 0x7f1d3c1330c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:42304/0 (socket says 172.21.15.47:42304) 2024-09-15T21:56:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d41a4c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c06e5f0 0x7f1d3c133600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d4224d640 1 -- 172.21.15.47:0/1658924377 learned_addr learned my addr 172.21.15.47:0/1658924377 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:58.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d42a4e640 1 --2- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c094940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d4224d640 1 -- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 msgr2=0x7f1d3c094940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d4224d640 1 --2- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c094940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d4224d640 1 -- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c06e5f0 msgr2=0x7f1d3c133600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d4224d640 1 --2- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c06e5f0 0x7f1d3c133600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d4224d640 1 -- 172.21.15.47:0/1658924377 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c133b40 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d41a4c640 1 --2- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c06e5f0 0x7f1d3c133600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d42a4e640 1 --2- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c094940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d4224d640 1 --2- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06dcd0 0x7f1d3c1330c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800e980 tx=0x7f1d3800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.959+0000 7f1d2effd640 1 -- 172.21.15.47:0/1658924377 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800ca00 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.960+0000 7f1d2effd640 1 -- 172.21.15.47:0/1658924377 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3800cba0 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.960+0000 7f1d2effd640 1 -- 172.21.15.47:0/1658924377 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38004030 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.960+0000 7f1d444b1640 1 -- 172.21.15.47:0/1658924377 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3c096bc0 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.960+0000 7f1d444b1640 1 -- 172.21.15.47:0/1658924377 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3c098000 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.961+0000 7f1d2effd640 1 -- 172.21.15.47:0/1658924377 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f1d380041d0 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.961+0000 7f1d444b1640 1 -- 172.21.15.47:0/1658924377 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d3c06e0b0 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.961+0000 7f1d2effd640 1 -- 172.21.15.47:0/1658924377 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f1d3805d960 con 0x7f1d3c06dcd0 2024-09-15T21:56:58.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:58.965+0000 7f1d2effd640 1 -- 172.21.15.47:0/1658924377 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f1d38017020 con 0x7f1d3c06dcd0 2024-09-15T21:56:59.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.109+0000 7f1d444b1640 1 -- 172.21.15.47:0/1658924377 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d3c068de0 con 0x7f1d20047fd0 2024-09-15T21:56:59.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.111+0000 7f1d2effd640 1 -- 172.21.15.47:0/1658924377 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32004 (unknown 2965378022 0 2052357454) 0x7f1d3c068de0 con 0x7f1d20047fd0 2024-09-15T21:56:59.128 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.113+0000 7f1d444b1640 1 -- 172.21.15.47:0/1658924377 >> v1:172.21.15.47:6800/1790418198 conn(0x7f1d20047fd0 legacy=0x7f1d2004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.113+0000 7f1d444b1640 1 -- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06dcd0 msgr2=0x7f1d3c1330c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.113+0000 7f1d444b1640 1 --2- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06dcd0 0x7f1d3c1330c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800e980 tx=0x7f1d3800ee40 comp rx=0 tx=0).stop 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.113+0000 7f1d444b1640 1 -- 172.21.15.47:0/1658924377 shutdown_connections 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.113+0000 7f1d444b1640 1 --2- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f1d3c133d10 0x7f1d3c094940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.113+0000 7f1d444b1640 1 --2- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c06e5f0 0x7f1d3c133600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.113+0000 7f1d444b1640 1 --2- 172.21.15.47:0/1658924377 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f1d3c06dcd0 0x7f1d3c1330c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.113+0000 7f1d444b1640 1 -- 172.21.15.47:0/1658924377 >> 172.21.15.47:0/1658924377 conn(0x7f1d3c09ed50 msgr2=0x7f1d3c09bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.113+0000 7f1d444b1640 1 -- 172.21.15.47:0/1658924377 shutdown_connections 2024-09-15T21:56:59.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.113+0000 7f1d444b1640 1 -- 172.21.15.47:0/1658924377 wait complete. 2024-09-15T21:56:59.131 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-15T21:56:57.676224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128956,"kb_used_data":2184,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358404,"statfs":{"total":386547056640,"available":386415005696,"internally_reserved":0,"allocated":2236416,"data_stored":1539721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001103"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:56:53.742497+0000","last_change":"2024-09-15T21:56:49.850959+0000","last_active":"2024-09-15T21:56:53.742497+0000","last_peered":"2024-09-15T21:56:53.742497+0000","last_clean":"2024-09-15T21:56:53.742497+0000","last_became_active":"2024-09-15T21:56:49.850607+0000","last_became_peered":"2024-09-15T21:56:49.850607+0000","last_unstale":"2024-09-15T21:56:53.742497+0000","last_undegraded":"2024-09-15T21:56:53.742497+0000","last_fullsized":"2024-09-15T21:56:53.742497+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.697372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743750+0000","last_change":"2024-09-15T21:56:47.710370+0000","last_active":"2024-09-15T21:56:54.743750+0000","last_peered":"2024-09-15T21:56:54.743750+0000","last_clean":"2024-09-15T21:56:54.743750+0000","last_became_active":"2024-09-15T21:56:47.710067+0000","last_became_peered":"2024-09-15T21:56:47.710067+0000","last_unstale":"2024-09-15T21:56:54.743750+0000","last_undegraded":"2024-09-15T21:56:54.743750+0000","last_fullsized":"2024-09-15T21:56:54.743750+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:50.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:56:53.742513+0000","last_change":"2024-09-15T21:56:49.851508+0000","last_active":"2024-09-15T21:56:53.742513+0000","last_peered":"2024-09-15T21:56:53.742513+0000","last_clean":"2024-09-15T21:56:53.742513+0000","last_became_active":"2024-09-15T21:56:49.851081+0000","last_became_peered":"2024-09-15T21:56:49.851081+0000","last_unstale":"2024-09-15T21:56:53.742513+0000","last_undegraded":"2024-09-15T21:56:53.742513+0000","last_fullsized":"2024-09-15T21:56:53.742513+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:39:15.989628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743740+0000","last_change":"2024-09-15T21:56:47.711356+0000","last_active":"2024-09-15T21:56:54.743740+0000","last_peered":"2024-09-15T21:56:54.743740+0000","last_clean":"2024-09-15T21:56:54.743740+0000","last_became_active":"2024-09-15T21:56:47.711012+0000","last_became_peered":"2024-09-15T21:56:47.711012+0000","last_unstale":"2024-09-15T21:56:54.743740+0000","last_undegraded":"2024-09-15T21:56:54.743740+0000","last_fullsized":"2024-09-15T21:56:54.743740+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:58:23.626385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":42,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:56:53.742469+0000","last_change":"2024-09-15T21:56:49.733169+0000","last_active":"2024-09-15T21:56:53.742469+0000","last_peered":"2024-09-15T21:56:53.742469+0000","last_clean":"2024-09-15T21:56:53.742469+0000","last_became_active":"2024-09-15T21:56:49.732754+0000","last_became_peered":"2024-09-15T21:56:49.732754+0000","last_unstale":"2024-09-15T21:56:53.742469+0000","last_undegraded":"2024-09-15T21:56:53.742469+0000","last_fullsized":"2024-09-15T21:56:53.742469+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:22.450704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743733+0000","last_change":"2024-09-15T21:56:49.733734+0000","last_active":"2024-09-15T21:56:54.743733+0000","last_peered":"2024-09-15T21:56:54.743733+0000","last_clean":"2024-09-15T21:56:54.743733+0000","last_became_active":"2024-09-15T21:56:49.733153+0000","last_became_peered":"2024-09-15T21:56:49.733153+0000","last_unstale":"2024-09-15T21:56:54.743733+0000","last_undegraded":"2024-09-15T21:56:54.743733+0000","last_fullsized":"2024-09-15T21:56:54.743733+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:56.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:56:53.742439+0000","last_change":"2024-09-15T21:56:49.731747+0000","last_active":"2024-09-15T21:56:53.742439+0000","last_peered":"2024-09-15T21:56:53.742439+0000","last_clean":"2024-09-15T21:56:53.742439+0000","last_became_active":"2024-09-15T21:56:49.731371+0000","last_became_peered":"2024-09-15T21:56:49.731371+0000","last_unstale":"2024-09-15T21:56:53.742439+0000","last_undegraded":"2024-09-15T21:56:53.742439+0000","last_fullsized":"2024-09-15T21:56:53.742439+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:49.586071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:56:53.742479+0000","last_change":"2024-09-15T21:56:49.733242+0000","last_active":"2024-09-15T21:56:53.742479+0000","last_peered":"2024-09-15T21:56:53.742479+0000","last_clean":"2024-09-15T21:56:53.742479+0000","last_became_active":"2024-09-15T21:56:49.732855+0000","last_became_peered":"2024-09-15T21:56:49.732855+0000","last_unstale":"2024-09-15T21:56:53.742479+0000","last_undegraded":"2024-09-15T21:56:53.742479+0000","last_fullsized":"2024-09-15T21:56:53.742479+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:46.865494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":146,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743991+0000","last_change":"2024-09-15T21:56:47.710563+0000","last_active":"2024-09-15T21:56:54.743991+0000","last_peered":"2024-09-15T21:56:54.743991+0000","last_clean":"2024-09-15T21:56:54.743991+0000","last_became_active":"2024-09-15T21:56:47.710235+0000","last_became_peered":"2024-09-15T21:56:47.710235+0000","last_unstale":"2024-09-15T21:56:54.743991+0000","last_undegraded":"2024-09-15T21:56:54.743991+0000","last_fullsized":"2024-09-15T21:56:54.743991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_clean_scrub_stamp":"2024-09-15T21:55:11.249936+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:43.977752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":27,"seq":115964116995,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":252,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":258048,"data_stored":87082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":27,"seq":115964116995,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":262144,"data_stored":90628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":840,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":860160,"data_stored":681015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":836,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":856064,"data_stored":680996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:56:59.131 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.230+0000 7fc3160cb640 1 Processor -- start 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.230+0000 7fc3160cb640 1 -- start start 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.230+0000 7fc3160cb640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310095370 0x7fc310095750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.230+0000 7fc3160cb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc310135030 0x7fc310137420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc3160cb640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc310096e10 con 0x7fc310095c90 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc3160cb640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc3100970f0 con 0x7fc310135030 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc3160cb640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fc310094b30 con 0x7fc310095370 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc30f7fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310095370 0x7fc310095750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc30effd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc310095c90 0x7fc3100960f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34410/0 (socket says 172.21.15.47:34410) 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc30effd640 1 -- 172.21.15.47:0/314596242 learned_addr learned my addr 172.21.15.47:0/314596242 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc30ffff640 1 --2- 172.21.15.47:0/314596242 >> v2:172.21.15.47:3300/0 conn(0x7fc310135030 0x7fc310137420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc30e7fc640 1 -- 172.21.15.47:0/314596242 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2852658431 0 0) 0x7fc304001110 con 0x7fc310095c90 2024-09-15T21:56:59.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc30ffff640 1 -- 172.21.15.47:0/314596242 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310095370 msgr2=0x7fc310095750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:59.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc30ffff640 1 --2- 172.21.15.47:0/314596242 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310095370 0x7fc310095750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc30ffff640 1 -- 172.21.15.47:0/314596242 >> v1:172.21.15.47:6789/0 conn(0x7fc310095c90 legacy=0x7fc3100960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc30ffff640 1 -- 172.21.15.47:0/314596242 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc310096800 con 0x7fc310135030 2024-09-15T21:56:59.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.231+0000 7fc30f7fe640 1 --2- 172.21.15.47:0/314596242 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310095370 0x7fc310095750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc30ffff640 1 --2- 172.21.15.47:0/314596242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310135030 0x7fc310137420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc308009d90 tx=0x7fc30802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c38b68dce01df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc30e7fc640 1 -- 172.21.15.47:0/314596242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308007740 con 0x7fc310135030 2024-09-15T21:56:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc30e7fc640 1 -- 172.21.15.47:0/314596242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3080078e0 con 0x7fc310135030 2024-09-15T21:56:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc30e7fc640 1 -- 172.21.15.47:0/314596242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308007cb0 con 0x7fc310135030 2024-09-15T21:56:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc3160cb640 1 -- 172.21.15.47:0/314596242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310135030 msgr2=0x7fc310137420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc3160cb640 1 --2- 172.21.15.47:0/314596242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310135030 0x7fc310137420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc308009d90 tx=0x7fc30802a530 comp rx=0 tx=0).stop 2024-09-15T21:56:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc3160cb640 1 -- 172.21.15.47:0/314596242 shutdown_connections 2024-09-15T21:56:59.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc3160cb640 1 --2- 172.21.15.47:0/314596242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310135030 0x7fc310137420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc3160cb640 1 --2- 172.21.15.47:0/314596242 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310095370 0x7fc310095750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc3160cb640 1 -- 172.21.15.47:0/314596242 >> 172.21.15.47:0/314596242 conn(0x7fc31009e050 msgr2=0x7fc31009e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc3160cb640 1 -- 172.21.15.47:0/314596242 shutdown_connections 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.232+0000 7fc3160cb640 1 -- 172.21.15.47:0/314596242 wait complete. 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.233+0000 7fc3160cb640 1 Processor -- start 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc3160cb640 1 -- start start 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc3160cb640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc310095370 0x7fc310122f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc3160cb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310095c90 0x7fc310123470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc3160cb640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310135030 0x7fc3101239b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc3160cb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc310137ae0 con 0x7fc310095c90 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc3160cb640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fc310137960 con 0x7fc310095370 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc3160cb640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fc310137c60 con 0x7fc310135030 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc30effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310095c90 0x7fc310123470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc30ffff640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310135030 0x7fc3101239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc30f7fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc310095370 0x7fc310122f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc30effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310095c90 0x7fc310123470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46130/0 (socket says 172.21.15.47:46130) 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc30effd640 1 -- 172.21.15.47:0/2333786564 learned_addr learned my addr 172.21.15.47:0/2333786564 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc30effd640 1 -- 172.21.15.47:0/2333786564 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310135030 msgr2=0x7fc3101239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc30effd640 1 --2- 172.21.15.47:0/2333786564 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310135030 0x7fc3101239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc30effd640 1 -- 172.21.15.47:0/2333786564 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc310095370 msgr2=0x7fc310122f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc30effd640 1 --2- 172.21.15.47:0/2333786564 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc310095370 0x7fc310122f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.234+0000 7fc30effd640 1 -- 172.21.15.47:0/2333786564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc310125340 con 0x7fc310095c90 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.235+0000 7fc30effd640 1 --2- 172.21.15.47:0/2333786564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310095c90 0x7fc310123470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc30400c780 tx=0x7fc30400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.235+0000 7fc2f7fff640 1 -- 172.21.15.47:0/2333786564 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304018070 con 0x7fc310095c90 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.235+0000 7fc2f7fff640 1 -- 172.21.15.47:0/2333786564 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3040072d0 con 0x7fc310095c90 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.235+0000 7fc3160cb640 1 -- 172.21.15.47:0/2333786564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3101241f0 con 0x7fc310095c90 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.235+0000 7fc2f7fff640 1 -- 172.21.15.47:0/2333786564 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040047e0 con 0x7fc310095c90 2024-09-15T21:56:59.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.235+0000 7fc3160cb640 1 -- 172.21.15.47:0/2333786564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc310073600 con 0x7fc310095c90 2024-09-15T21:56:59.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.236+0000 7fc2f7fff640 1 -- 172.21.15.47:0/2333786564 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fc304004980 con 0x7fc310095c90 2024-09-15T21:56:59.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.236+0000 7fc3160cb640 1 -- 172.21.15.47:0/2333786564 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc310073890 con 0x7fc310095c90 2024-09-15T21:56:59.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.236+0000 7fc2f7fff640 1 -- 172.21.15.47:0/2333786564 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fc30405cd60 con 0x7fc310095c90 2024-09-15T21:56:59.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.240+0000 7fc2f7fff640 1 -- 172.21.15.47:0/2333786564 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fc310073890 con 0x7fc310095c90 2024-09-15T21:56:59.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.382+0000 7fc3160cb640 1 -- 172.21.15.47:0/2333786564 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc310095750 con 0x7fc2fc047fd0 2024-09-15T21:56:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.383+0000 7fc2f7fff640 1 -- 172.21.15.47:0/2333786564 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32004 (unknown 2965378022 0 2052357454) 0x7fc310095750 con 0x7fc2fc047fd0 2024-09-15T21:56:59.386 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:59.386 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:56:59.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.384+0000 7fc3160cb640 1 -- 172.21.15.47:0/2333786564 >> v1:172.21.15.47:6800/1790418198 conn(0x7fc2fc047fd0 legacy=0x7fc2fc04a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.384+0000 7fc3160cb640 1 -- 172.21.15.47:0/2333786564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310095c90 msgr2=0x7fc310123470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.384+0000 7fc3160cb640 1 --2- 172.21.15.47:0/2333786564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310095c90 0x7fc310123470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc30400c780 tx=0x7fc30400cc40 comp rx=0 tx=0).stop 2024-09-15T21:56:59.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.385+0000 7fc3160cb640 1 -- 172.21.15.47:0/2333786564 shutdown_connections 2024-09-15T21:56:59.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.385+0000 7fc3160cb640 1 --2- 172.21.15.47:0/2333786564 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc310135030 0x7fc3101239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.385+0000 7fc3160cb640 1 --2- 172.21.15.47:0/2333786564 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc310095c90 0x7fc310123470 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.385+0000 7fc3160cb640 1 --2- 172.21.15.47:0/2333786564 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc310095370 0x7fc310122f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.385+0000 7fc3160cb640 1 -- 172.21.15.47:0/2333786564 >> 172.21.15.47:0/2333786564 conn(0x7fc31009e050 msgr2=0x7fc3100992a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:59.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.385+0000 7fc3160cb640 1 -- 172.21.15.47:0/2333786564 shutdown_connections 2024-09-15T21:56:59.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.385+0000 7fc3160cb640 1 -- 172.21.15.47:0/2333786564 wait complete. 2024-09-15T21:56:59.398 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-15T21:56:57.676224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128956,"kb_used_data":2184,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358404,"statfs":{"total":386547056640,"available":386415005696,"internally_reserved":0,"allocated":2236416,"data_stored":1539721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001103"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:56:53.742497+0000","last_change":"2024-09-15T21:56:49.850959+0000","last_active":"2024-09-15T21:56:53.742497+0000","last_peered":"2024-09-15T21:56:53.742497+0000","last_clean":"2024-09-15T21:56:53.742497+0000","last_became_active":"2024-09-15T21:56:49.850607+0000","last_became_peered":"2024-09-15T21:56:49.850607+0000","last_unstale":"2024-09-15T21:56:53.742497+0000","last_undegraded":"2024-09-15T21:56:53.742497+0000","last_fullsized":"2024-09-15T21:56:53.742497+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.697372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743750+0000","last_change":"2024-09-15T21:56:47.710370+0000","last_active":"2024-09-15T21:56:54.743750+0000","last_peered":"2024-09-15T21:56:54.743750+0000","last_clean":"2024-09-15T21:56:54.743750+0000","last_became_active":"2024-09-15T21:56:47.710067+0000","last_became_peered":"2024-09-15T21:56:47.710067+0000","last_unstale":"2024-09-15T21:56:54.743750+0000","last_undegraded":"2024-09-15T21:56:54.743750+0000","last_fullsized":"2024-09-15T21:56:54.743750+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:50.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:56:53.742513+0000","last_change":"2024-09-15T21:56:49.851508+0000","last_active":"2024-09-15T21:56:53.742513+0000","last_peered":"2024-09-15T21:56:53.742513+0000","last_clean":"2024-09-15T21:56:53.742513+0000","last_became_active":"2024-09-15T21:56:49.851081+0000","last_became_peered":"2024-09-15T21:56:49.851081+0000","last_unstale":"2024-09-15T21:56:53.742513+0000","last_undegraded":"2024-09-15T21:56:53.742513+0000","last_fullsized":"2024-09-15T21:56:53.742513+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:39:15.989628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743740+0000","last_change":"2024-09-15T21:56:47.711356+0000","last_active":"2024-09-15T21:56:54.743740+0000","last_peered":"2024-09-15T21:56:54.743740+0000","last_clean":"2024-09-15T21:56:54.743740+0000","last_became_active":"2024-09-15T21:56:47.711012+0000","last_became_peered":"2024-09-15T21:56:47.711012+0000","last_unstale":"2024-09-15T21:56:54.743740+0000","last_undegraded":"2024-09-15T21:56:54.743740+0000","last_fullsized":"2024-09-15T21:56:54.743740+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:58:23.626385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":42,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:56:53.742469+0000","last_change":"2024-09-15T21:56:49.733169+0000","last_active":"2024-09-15T21:56:53.742469+0000","last_peered":"2024-09-15T21:56:53.742469+0000","last_clean":"2024-09-15T21:56:53.742469+0000","last_became_active":"2024-09-15T21:56:49.732754+0000","last_became_peered":"2024-09-15T21:56:49.732754+0000","last_unstale":"2024-09-15T21:56:53.742469+0000","last_undegraded":"2024-09-15T21:56:53.742469+0000","last_fullsized":"2024-09-15T21:56:53.742469+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:56:22.450704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743733+0000","last_change":"2024-09-15T21:56:49.733734+0000","last_active":"2024-09-15T21:56:54.743733+0000","last_peered":"2024-09-15T21:56:54.743733+0000","last_clean":"2024-09-15T21:56:54.743733+0000","last_became_active":"2024-09-15T21:56:49.733153+0000","last_became_peered":"2024-09-15T21:56:49.733153+0000","last_unstale":"2024-09-15T21:56:54.743733+0000","last_undegraded":"2024-09-15T21:56:54.743733+0000","last_fullsized":"2024-09-15T21:56:54.743733+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:56.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:56:53.742439+0000","last_change":"2024-09-15T21:56:49.731747+0000","last_active":"2024-09-15T21:56:53.742439+0000","last_peered":"2024-09-15T21:56:53.742439+0000","last_clean":"2024-09-15T21:56:53.742439+0000","last_became_active":"2024-09-15T21:56:49.731371+0000","last_became_peered":"2024-09-15T21:56:49.731371+0000","last_unstale":"2024-09-15T21:56:53.742439+0000","last_undegraded":"2024-09-15T21:56:53.742439+0000","last_fullsized":"2024-09-15T21:56:53.742439+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:16:49.586071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:56:53.742479+0000","last_change":"2024-09-15T21:56:49.733242+0000","last_active":"2024-09-15T21:56:53.742479+0000","last_peered":"2024-09-15T21:56:53.742479+0000","last_clean":"2024-09-15T21:56:53.742479+0000","last_became_active":"2024-09-15T21:56:49.732855+0000","last_became_peered":"2024-09-15T21:56:49.732855+0000","last_unstale":"2024-09-15T21:56:53.742479+0000","last_undegraded":"2024-09-15T21:56:53.742479+0000","last_fullsized":"2024-09-15T21:56:53.742479+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:46.865494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":146,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743991+0000","last_change":"2024-09-15T21:56:47.710563+0000","last_active":"2024-09-15T21:56:54.743991+0000","last_peered":"2024-09-15T21:56:54.743991+0000","last_clean":"2024-09-15T21:56:54.743991+0000","last_became_active":"2024-09-15T21:56:47.710235+0000","last_became_peered":"2024-09-15T21:56:47.710235+0000","last_unstale":"2024-09-15T21:56:54.743991+0000","last_undegraded":"2024-09-15T21:56:54.743991+0000","last_fullsized":"2024-09-15T21:56:54.743991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_clean_scrub_stamp":"2024-09-15T21:55:11.249936+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:43.977752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":27,"seq":115964116995,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":252,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":258048,"data_stored":87082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":27,"seq":115964116995,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":262144,"data_stored":90628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":840,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":860160,"data_stored":681015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":836,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":856064,"data_stored":680996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:56:59.399 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:56:59.399 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T21:56:59.399 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T21:56:59.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:56:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.496+0000 7f9f33383640 1 Processor -- start 2024-09-15T21:56:59.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.496+0000 7f9f33383640 1 -- start start 2024-09-15T21:56:59.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f33383640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c09a630 0x7f9f2c09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f33383640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c12faf0 0x7f9f2c12ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f33383640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2c09b600 con 0x7f9f2c09af50 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f33383640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c06d480 con 0x7f9f2c09a630 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f33383640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f9f2c06d140 con 0x7f9f2c12faf0 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f3091e640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9f2c09af50 0x7f9f2c12f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34210/0 (socket says 172.21.15.47:34210) 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f3091e640 1 -- 172.21.15.47:0/1171256125 learned_addr learned my addr 172.21.15.47:0/1171256125 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f3111f640 1 --2- 172.21.15.47:0/1171256125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c09a630 0x7f9f2c09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f31920640 1 --2- 172.21.15.47:0/1171256125 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c12faf0 0x7f9f2c12ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f23fff640 1 -- 172.21.15.47:0/1171256125 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 790185040 0 0) 0x7f9f2c09b600 con 0x7f9f2c09af50 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f31920640 1 -- 172.21.15.47:0/1171256125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c09a630 msgr2=0x7f9f2c09aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f31920640 1 --2- 172.21.15.47:0/1171256125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c09a630 0x7f9f2c09aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f31920640 1 -- 172.21.15.47:0/1171256125 >> v1:172.21.15.47:6789/0 conn(0x7f9f2c09af50 legacy=0x7f9f2c12f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f31920640 1 -- 172.21.15.47:0/1171256125 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c09bc00 con 0x7f9f2c12faf0 2024-09-15T21:56:59.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.497+0000 7f9f3111f640 1 --2- 172.21.15.47:0/1171256125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c09a630 0x7f9f2c09aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:59.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.498+0000 7f9f31920640 1 --2- 172.21.15.47:0/1171256125 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c12faf0 0x7f9f2c12ff80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009dc0 tx=0x7f9f2402a510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7558a1044605996a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:59.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.498+0000 7f9f23fff640 1 -- 172.21.15.47:0/1171256125 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24007740 con 0x7f9f2c12faf0 2024-09-15T21:56:59.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.498+0000 7f9f23fff640 1 -- 172.21.15.47:0/1171256125 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f240078e0 con 0x7f9f2c12faf0 2024-09-15T21:56:59.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.498+0000 7f9f23fff640 1 -- 172.21.15.47:0/1171256125 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24007cb0 con 0x7f9f2c12faf0 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.499+0000 7f9f33383640 1 -- 172.21.15.47:0/1171256125 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c12faf0 msgr2=0x7f9f2c12ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.499+0000 7f9f33383640 1 --2- 172.21.15.47:0/1171256125 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c12faf0 0x7f9f2c12ff80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009dc0 tx=0x7f9f2402a510 comp rx=0 tx=0).stop 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.499+0000 7f9f33383640 1 -- 172.21.15.47:0/1171256125 shutdown_connections 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.499+0000 7f9f33383640 1 --2- 172.21.15.47:0/1171256125 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c12faf0 0x7f9f2c12ff80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.499+0000 7f9f33383640 1 --2- 172.21.15.47:0/1171256125 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c09a630 0x7f9f2c09aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.499+0000 7f9f33383640 1 -- 172.21.15.47:0/1171256125 >> 172.21.15.47:0/1171256125 conn(0x7f9f2c09eb90 msgr2=0x7f9f2c09efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.499+0000 7f9f33383640 1 -- 172.21.15.47:0/1171256125 shutdown_connections 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.499+0000 7f9f33383640 1 -- 172.21.15.47:0/1171256125 wait complete. 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f33383640 1 Processor -- start 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f33383640 1 -- start start 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f33383640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2c09a630 0x7f9f2c128850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f33383640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c09af50 0x7f9f2c128d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f33383640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c12faf0 0x7f9f2c1292d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f33383640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c137ad0 con 0x7f9f2c09a630 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f33383640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c137950 con 0x7f9f2c12faf0 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f33383640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f9f2c137c50 con 0x7f9f2c09af50 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f3111f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2c09a630 0x7f9f2c128850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f3111f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2c09a630 0x7f9f2c128850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57854/0 (socket says 172.21.15.47:57854) 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f3111f640 1 -- 172.21.15.47:0/2560488713 learned_addr learned my addr 172.21.15.47:0/2560488713 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f31920640 1 --2- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c12faf0 0x7f9f2c1292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.500+0000 7f9f3091e640 1 --2- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c09af50 0x7f9f2c128d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f31920640 1 -- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c09af50 msgr2=0x7f9f2c128d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f31920640 1 --2- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c09af50 0x7f9f2c128d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f31920640 1 -- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2c09a630 msgr2=0x7f9f2c128850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f31920640 1 --2- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2c09a630 0x7f9f2c128850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f31920640 1 -- 172.21.15.47:0/2560488713 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c12ab30 con 0x7f9f2c12faf0 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f3111f640 1 --2- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2c09a630 0x7f9f2c128850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f31920640 1 --2- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c12faf0 0x7f9f2c1292d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f24031000 tx=0x7f9f24004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f21ffb640 1 -- 172.21.15.47:0/2560488713 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24040070 con 0x7f9f2c12faf0 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f21ffb640 1 -- 172.21.15.47:0/2560488713 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f24031bc0 con 0x7f9f2c12faf0 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f33383640 1 -- 172.21.15.47:0/2560488713 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c12b6c0 con 0x7f9f2c12faf0 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f21ffb640 1 -- 172.21.15.47:0/2560488713 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24009450 con 0x7f9f2c12faf0 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.501+0000 7f9f33383640 1 -- 172.21.15.47:0/2560488713 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c12b3e0 con 0x7f9f2c12faf0 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.502+0000 7f9f21ffb640 1 -- 172.21.15.47:0/2560488713 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f9f24039040 con 0x7f9f2c12faf0 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.502+0000 7f9f33383640 1 -- 172.21.15.47:0/2560488713 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2c12c0d0 con 0x7f9f2c12faf0 2024-09-15T21:56:59.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.502+0000 7f9f21ffb640 1 -- 172.21.15.47:0/2560488713 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f9f2407ea50 con 0x7f9f2c12faf0 2024-09-15T21:56:59.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.506+0000 7f9f21ffb640 1 -- 172.21.15.47:0/2560488713 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f9f2c12b6c0 con 0x7f9f2c12faf0 2024-09-15T21:56:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.679+0000 7f9f33383640 1 -- 172.21.15.47:0/2560488713 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9f2c06d840 con 0x7f9f2c12faf0 2024-09-15T21:56:59.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.679+0000 7f9f21ffb640 1 -- 172.21.15.47:0/2560488713 <== mon.1 v2:172.21.15.47:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9f2c06d840 con 0x7f9f2c12faf0 2024-09-15T21:56:59.682 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:56:59.682 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T21:56:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.681+0000 7f9f33383640 1 -- 172.21.15.47:0/2560488713 >> v1:172.21.15.47:6800/1790418198 conn(0x7f9f10047fd0 legacy=0x7f9f1004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.681+0000 7f9f33383640 1 -- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c12faf0 msgr2=0x7f9f2c1292d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.681+0000 7f9f33383640 1 --2- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c12faf0 0x7f9f2c1292d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f24031000 tx=0x7f9f24004a40 comp rx=0 tx=0).stop 2024-09-15T21:56:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.681+0000 7f9f33383640 1 -- 172.21.15.47:0/2560488713 shutdown_connections 2024-09-15T21:56:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.681+0000 7f9f33383640 1 --2- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f9f2c12faf0 0x7f9f2c1292d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.681+0000 7f9f33383640 1 --2- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f9f2c09af50 0x7f9f2c128d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.681+0000 7f9f33383640 1 --2- 172.21.15.47:0/2560488713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9f2c09a630 0x7f9f2c128850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.681+0000 7f9f33383640 1 -- 172.21.15.47:0/2560488713 >> 172.21.15.47:0/2560488713 conn(0x7f9f2c09eb90 msgr2=0x7f9f2c09d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:59.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.681+0000 7f9f33383640 1 -- 172.21.15.47:0/2560488713 shutdown_connections 2024-09-15T21:56:59.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.681+0000 7f9f33383640 1 -- 172.21.15.47:0/2560488713 wait complete. 2024-09-15T21:56:59.695 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T21:56:59.695 INFO:teuthology.run_tasks:Running task print... 2024-09-15T21:56:59.705 INFO:teuthology.task.print:**** done ceph.restart 2024-09-15T21:56:59.706 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2024-09-15T21:56:59.714 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2024-09-15T21:56:59.723 DEBUG:teuthology.run_tasks:Unwinding manager install.upgrade 2024-09-15T21:56:59.734 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T21:56:59.743 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:56:59.744 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:56:59.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.844+0000 7f7090c8a640 1 Processor -- start 2024-09-15T21:56:59.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.844+0000 7f7090c8a640 1 -- start start 2024-09-15T21:56:59.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f7090c8a640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c12f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f7090c8a640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f708c12fce0 0x7f708c098090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f7090c8a640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c0985d0 0x7f708c098a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f7090c8a640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f708c06f260 con 0x7f708c12fce0 2024-09-15T21:56:59.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f7090c8a640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f708c099600 con 0x7f708c0985d0 2024-09-15T21:56:59.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f7090c8a640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f708c09f230 con 0x7f708c12f3c0 2024-09-15T21:56:59.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f708a59c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c12f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f7089d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f708c12fce0 0x7f708c098090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f708ad9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c0985d0 0x7f708c098a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f708a59c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c12f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36902/0 (socket says 172.21.15.47:36902) 2024-09-15T21:56:59.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f708a59c640 1 -- 172.21.15.47:0/2659701986 learned_addr learned my addr 172.21.15.47:0/2659701986 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:59.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f7089d9b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f708c12fce0 0x7f708c098090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57866/0 (socket says 172.21.15.47:57866) 2024-09-15T21:56:59.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f708a59c640 1 -- 172.21.15.47:0/2659701986 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c0985d0 msgr2=0x7f708c098a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:59.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f708a59c640 1 --2- 172.21.15.47:0/2659701986 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c0985d0 0x7f708c098a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f708a59c640 1 -- 172.21.15.47:0/2659701986 >> v2:172.21.15.47:3300/0 conn(0x7f708c12fce0 msgr2=0x7f708c098090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:56:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f708a59c640 1 --2- 172.21.15.47:0/2659701986 >> v2:172.21.15.47:3300/0 conn(0x7f708c12fce0 0x7f708c098090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f708a59c640 1 -- 172.21.15.47:0/2659701986 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c130120 con 0x7f708c12f3c0 2024-09-15T21:56:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.845+0000 7f7089d9b640 1 --2- 172.21.15.47:0/2659701986 >> v2:172.21.15.47:3300/0 conn(0x7f708c12fce0 0x7f708c098090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.846+0000 7f708a59c640 1 --2- 172.21.15.47:0/2659701986 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c12f7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f707c009870 tx=0x7f707c02a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d18198d6e0af9a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.846+0000 7f708959a640 1 -- 172.21.15.47:0/2659701986 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c037070 con 0x7f708c12f3c0 2024-09-15T21:56:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.846+0000 7f708959a640 1 -- 172.21.15.47:0/2659701986 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707c02adf0 con 0x7f708c12f3c0 2024-09-15T21:56:59.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.846+0000 7f708959a640 1 -- 172.21.15.47:0/2659701986 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c032660 con 0x7f708c12f3c0 2024-09-15T21:56:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.847+0000 7f7090c8a640 1 -- 172.21.15.47:0/2659701986 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 msgr2=0x7f708c12f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.847+0000 7f7090c8a640 1 --2- 172.21.15.47:0/2659701986 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c12f7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f707c009870 tx=0x7f707c02a0a0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.847+0000 7f7090c8a640 1 -- 172.21.15.47:0/2659701986 shutdown_connections 2024-09-15T21:56:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.847+0000 7f7090c8a640 1 --2- 172.21.15.47:0/2659701986 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c0985d0 0x7f708c098a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.847+0000 7f7090c8a640 1 --2- 172.21.15.47:0/2659701986 >> v2:172.21.15.47:3300/0 conn(0x7f708c12fce0 0x7f708c098090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.847+0000 7f7090c8a640 1 --2- 172.21.15.47:0/2659701986 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c12f7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.847+0000 7f7090c8a640 1 -- 172.21.15.47:0/2659701986 >> 172.21.15.47:0/2659701986 conn(0x7f708c0a23b0 msgr2=0x7f708c09e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:56:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.847+0000 7f7090c8a640 1 -- 172.21.15.47:0/2659701986 shutdown_connections 2024-09-15T21:56:59.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.847+0000 7f7090c8a640 1 -- 172.21.15.47:0/2659701986 wait complete. 2024-09-15T21:56:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f7090c8a640 1 Processor -- start 2024-09-15T21:56:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f7090c8a640 1 -- start start 2024-09-15T21:56:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f7090c8a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f708c0985d0 0x7f708c1270e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f7090c8a640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c127620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f7090c8a640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c12fce0 0x7f708c127b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:56:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f7090c8a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f708c136920 con 0x7f708c0985d0 2024-09-15T21:56:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f7090c8a640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f708c06ee60 con 0x7f708c12fce0 2024-09-15T21:56:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f7090c8a640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f708c136aa0 con 0x7f708c12f3c0 2024-09-15T21:56:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f708a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f708c0985d0 0x7f708c1270e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f708a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f708c0985d0 0x7f708c1270e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57878/0 (socket says 172.21.15.47:57878) 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f7089d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c127620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f708a59c640 1 -- 172.21.15.47:0/3308558966 learned_addr learned my addr 172.21.15.47:0/3308558966 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.848+0000 7f708ad9d640 1 --2- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c12fce0 0x7f708c127b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f708a59c640 1 -- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 msgr2=0x7f708c127620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f708a59c640 1 --2- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c127620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f708a59c640 1 -- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c12fce0 msgr2=0x7f708c127b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f708a59c640 1 --2- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c12fce0 0x7f708c127b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f708a59c640 1 -- 172.21.15.47:0/3308558966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c12a4c0 con 0x7f708c0985d0 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f708ad9d640 1 --2- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c12fce0 0x7f708c127b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f7089d9b640 1 --2- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c127620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f708a59c640 1 --2- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f708c0985d0 0x7f708c1270e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c00 tx=0x7f707c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:56:59.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f707affd640 1 -- 172.21.15.47:0/3308558966 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c041020 con 0x7f708c0985d0 2024-09-15T21:56:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f707affd640 1 -- 172.21.15.47:0/3308558966 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707c03b400 con 0x7f708c0985d0 2024-09-15T21:56:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f7090c8a640 1 -- 172.21.15.47:0/3308558966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708c12cd60 con 0x7f708c0985d0 2024-09-15T21:56:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f707affd640 1 -- 172.21.15.47:0/3308558966 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c037040 con 0x7f708c0985d0 2024-09-15T21:56:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.849+0000 7f7090c8a640 1 -- 172.21.15.47:0/3308558966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708c1ca9f0 con 0x7f708c0985d0 2024-09-15T21:56:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.850+0000 7f7090c8a640 1 -- 172.21.15.47:0/3308558966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708c12f7a0 con 0x7f708c0985d0 2024-09-15T21:56:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.850+0000 7f707affd640 1 -- 172.21.15.47:0/3308558966 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f707c044020 con 0x7f708c0985d0 2024-09-15T21:56:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.851+0000 7f707affd640 1 -- 172.21.15.47:0/3308558966 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f707c07d880 con 0x7f708c0985d0 2024-09-15T21:56:59.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.854+0000 7f707affd640 1 -- 172.21.15.47:0/3308558966 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f707c04b030 con 0x7f708c0985d0 2024-09-15T21:57:00.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.997+0000 7f7090c8a640 1 -- 172.21.15.47:0/3308558966 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f708c1288a0 con 0x7f706c048000 2024-09-15T21:57:00.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:56:59.998+0000 7f707affd640 1 -- 172.21.15.47:0/3308558966 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32004 (unknown 2965378022 0 1661938822) 0x7f708c1288a0 con 0x7f706c048000 2024-09-15T21:57:00.001 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:57:00.001 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:57:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.000+0000 7f7090c8a640 1 -- 172.21.15.47:0/3308558966 >> v1:172.21.15.47:6800/1790418198 conn(0x7f706c048000 legacy=0x7f706c04a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.000+0000 7f7090c8a640 1 -- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f708c0985d0 msgr2=0x7f708c1270e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.000+0000 7f7090c8a640 1 --2- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f708c0985d0 0x7f708c1270e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c00 tx=0x7f707c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.000+0000 7f7090c8a640 1 -- 172.21.15.47:0/3308558966 shutdown_connections 2024-09-15T21:57:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.000+0000 7f7090c8a640 1 --2- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f708c12fce0 0x7f708c127b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.000+0000 7f7090c8a640 1 --2- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f708c12f3c0 0x7f708c127620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.000+0000 7f7090c8a640 1 --2- 172.21.15.47:0/3308558966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f708c0985d0 0x7f708c1270e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.000+0000 7f7090c8a640 1 -- 172.21.15.47:0/3308558966 >> 172.21.15.47:0/3308558966 conn(0x7f708c0a23b0 msgr2=0x7f708c12e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.000+0000 7f7090c8a640 1 -- 172.21.15.47:0/3308558966 shutdown_connections 2024-09-15T21:57:00.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.000+0000 7f7090c8a640 1 -- 172.21.15.47:0/3308558966 wait complete. 2024-09-15T21:57:00.013 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-15T21:56:59.676524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128968,"kb_used_data":2192,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358392,"statfs":{"total":386547056640,"available":386414993408,"internally_reserved":0,"allocated":2244608,"data_stored":1543286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.943394"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185545+0000","last_change":"2024-09-15T21:56:49.850959+0000","last_active":"2024-09-15T21:56:56.185545+0000","last_peered":"2024-09-15T21:56:56.185545+0000","last_clean":"2024-09-15T21:56:56.185545+0000","last_became_active":"2024-09-15T21:56:49.850607+0000","last_became_peered":"2024-09-15T21:56:49.850607+0000","last_unstale":"2024-09-15T21:56:56.185545+0000","last_undegraded":"2024-09-15T21:56:56.185545+0000","last_fullsized":"2024-09-15T21:56:56.185545+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.532841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743750+0000","last_change":"2024-09-15T21:56:47.710370+0000","last_active":"2024-09-15T21:56:54.743750+0000","last_peered":"2024-09-15T21:56:54.743750+0000","last_clean":"2024-09-15T21:56:54.743750+0000","last_became_active":"2024-09-15T21:56:47.710067+0000","last_became_peered":"2024-09-15T21:56:47.710067+0000","last_unstale":"2024-09-15T21:56:54.743750+0000","last_undegraded":"2024-09-15T21:56:54.743750+0000","last_fullsized":"2024-09-15T21:56:54.743750+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:50.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185549+0000","last_change":"2024-09-15T21:56:49.851508+0000","last_active":"2024-09-15T21:56:56.185549+0000","last_peered":"2024-09-15T21:56:56.185549+0000","last_clean":"2024-09-15T21:56:56.185549+0000","last_became_active":"2024-09-15T21:56:49.851081+0000","last_became_peered":"2024-09-15T21:56:49.851081+0000","last_unstale":"2024-09-15T21:56:56.185549+0000","last_undegraded":"2024-09-15T21:56:56.185549+0000","last_fullsized":"2024-09-15T21:56:56.185549+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:03.698746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743740+0000","last_change":"2024-09-15T21:56:47.711356+0000","last_active":"2024-09-15T21:56:54.743740+0000","last_peered":"2024-09-15T21:56:54.743740+0000","last_clean":"2024-09-15T21:56:54.743740+0000","last_became_active":"2024-09-15T21:56:47.711012+0000","last_became_peered":"2024-09-15T21:56:47.711012+0000","last_unstale":"2024-09-15T21:56:54.743740+0000","last_undegraded":"2024-09-15T21:56:54.743740+0000","last_fullsized":"2024-09-15T21:56:54.743740+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:58:23.626385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185508+0000","last_change":"2024-09-15T21:56:49.733169+0000","last_active":"2024-09-15T21:56:56.185508+0000","last_peered":"2024-09-15T21:56:56.185508+0000","last_clean":"2024-09-15T21:56:56.185508+0000","last_became_active":"2024-09-15T21:56:49.732754+0000","last_became_peered":"2024-09-15T21:56:49.732754+0000","last_unstale":"2024-09-15T21:56:56.185508+0000","last_undegraded":"2024-09-15T21:56:56.185508+0000","last_fullsized":"2024-09-15T21:56:56.185508+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:11.761893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743733+0000","last_change":"2024-09-15T21:56:49.733734+0000","last_active":"2024-09-15T21:56:54.743733+0000","last_peered":"2024-09-15T21:56:54.743733+0000","last_clean":"2024-09-15T21:56:54.743733+0000","last_became_active":"2024-09-15T21:56:49.733153+0000","last_became_peered":"2024-09-15T21:56:49.733153+0000","last_unstale":"2024-09-15T21:56:54.743733+0000","last_undegraded":"2024-09-15T21:56:54.743733+0000","last_fullsized":"2024-09-15T21:56:54.743733+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:56.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185503+0000","last_change":"2024-09-15T21:56:49.731747+0000","last_active":"2024-09-15T21:56:56.185503+0000","last_peered":"2024-09-15T21:56:56.185503+0000","last_clean":"2024-09-15T21:56:56.185503+0000","last_became_active":"2024-09-15T21:56:49.731371+0000","last_became_peered":"2024-09-15T21:56:49.731371+0000","last_unstale":"2024-09-15T21:56:56.185503+0000","last_undegraded":"2024-09-15T21:56:56.185503+0000","last_fullsized":"2024-09-15T21:56:56.185503+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:27.373825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185514+0000","last_change":"2024-09-15T21:56:49.733242+0000","last_active":"2024-09-15T21:56:56.185514+0000","last_peered":"2024-09-15T21:56:56.185514+0000","last_clean":"2024-09-15T21:56:56.185514+0000","last_became_active":"2024-09-15T21:56:49.732855+0000","last_became_peered":"2024-09-15T21:56:49.732855+0000","last_unstale":"2024-09-15T21:56:56.185514+0000","last_undegraded":"2024-09-15T21:56:56.185514+0000","last_fullsized":"2024-09-15T21:56:56.185514+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:50:43.371057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":146,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743991+0000","last_change":"2024-09-15T21:56:47.710563+0000","last_active":"2024-09-15T21:56:54.743991+0000","last_peered":"2024-09-15T21:56:54.743991+0000","last_clean":"2024-09-15T21:56:54.743991+0000","last_became_active":"2024-09-15T21:56:47.710235+0000","last_became_peered":"2024-09-15T21:56:47.710235+0000","last_unstale":"2024-09-15T21:56:54.743991+0000","last_undegraded":"2024-09-15T21:56:54.743991+0000","last_fullsized":"2024-09-15T21:56:54.743991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_clean_scrub_stamp":"2024-09-15T21:55:11.249936+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:43.977752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":27,"seq":115964116996,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":266240,"data_stored":90647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":27,"seq":115964116996,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":262144,"data_stored":90628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":840,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":860160,"data_stored":681015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":836,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":856064,"data_stored":680996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:57:00.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:57:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.114+0000 7ffae18b1640 1 Processor -- start 2024-09-15T21:57:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffae18b1640 1 -- start start 2024-09-15T21:57:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffae18b1640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc09b600 0x7ffadc12f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffae18b1640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc12ff10 0x7ffadc12bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffae18b1640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffadc12c500 0x7ffadc12c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffae18b1640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ffadc06fbd0 con 0x7ffadc12c500 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffae18b1640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7ffadc06d4c0 con 0x7ffadc09b600 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffae18b1640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7ffadc12f730 con 0x7ffadc12ff10 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffadb7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffadc12c500 0x7ffadc12c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffada7fc640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc12ff10 0x7ffadc12bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffada7fc640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc12ff10 0x7ffadc12bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36922/0 (socket says 172.21.15.47:36922) 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffada7fc640 1 -- 172.21.15.47:0/1960688714 learned_addr learned my addr 172.21.15.47:0/1960688714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffadaffd640 1 --2- 172.21.15.47:0/1960688714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc09b600 0x7ffadc12f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffada7fc640 1 -- 172.21.15.47:0/1960688714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc09b600 msgr2=0x7ffadc12f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:00.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffada7fc640 1 --2- 172.21.15.47:0/1960688714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc09b600 0x7ffadc12f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffada7fc640 1 -- 172.21.15.47:0/1960688714 >> v2:172.21.15.47:3300/0 conn(0x7ffadc12c500 msgr2=0x7ffadc12c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:00.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffada7fc640 1 --2- 172.21.15.47:0/1960688714 >> v2:172.21.15.47:3300/0 conn(0x7ffadc12c500 0x7ffadc12c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffada7fc640 1 -- 172.21.15.47:0/1960688714 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc130310 con 0x7ffadc12ff10 2024-09-15T21:57:00.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.115+0000 7ffadb7fe640 1 --2- 172.21.15.47:0/1960688714 >> v2:172.21.15.47:3300/0 conn(0x7ffadc12c500 0x7ffadc12c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.116+0000 7ffada7fc640 1 --2- 172.21.15.47:0/1960688714 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc12ff10 0x7ffadc12bfc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009870 tx=0x7ffad002a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26d99b2c154f09de server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.116+0000 7ffad9ffb640 1 -- 172.21.15.47:0/1960688714 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0037070 con 0x7ffadc12ff10 2024-09-15T21:57:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.116+0000 7ffad9ffb640 1 -- 172.21.15.47:0/1960688714 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad002adf0 con 0x7ffadc12ff10 2024-09-15T21:57:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.116+0000 7ffad9ffb640 1 -- 172.21.15.47:0/1960688714 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0032660 con 0x7ffadc12ff10 2024-09-15T21:57:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.117+0000 7ffae18b1640 1 -- 172.21.15.47:0/1960688714 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc12ff10 msgr2=0x7ffadc12bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.117+0000 7ffae18b1640 1 --2- 172.21.15.47:0/1960688714 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc12ff10 0x7ffadc12bfc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009870 tx=0x7ffad002a0a0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.117+0000 7ffae18b1640 1 -- 172.21.15.47:0/1960688714 shutdown_connections 2024-09-15T21:57:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.117+0000 7ffae18b1640 1 --2- 172.21.15.47:0/1960688714 >> v2:172.21.15.47:3300/0 conn(0x7ffadc12c500 0x7ffadc12c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.117+0000 7ffae18b1640 1 --2- 172.21.15.47:0/1960688714 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc12ff10 0x7ffadc12bfc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.117+0000 7ffae18b1640 1 --2- 172.21.15.47:0/1960688714 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc09b600 0x7ffadc12f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.117+0000 7ffae18b1640 1 -- 172.21.15.47:0/1960688714 >> 172.21.15.47:0/1960688714 conn(0x7ffadc09eb20 msgr2=0x7ffadc09ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.117+0000 7ffae18b1640 1 -- 172.21.15.47:0/1960688714 shutdown_connections 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.117+0000 7ffae18b1640 1 -- 172.21.15.47:0/1960688714 wait complete. 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.118+0000 7ffae18b1640 1 Processor -- start 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.118+0000 7ffae18b1640 1 -- start start 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.118+0000 7ffae18b1640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc09b600 0x7ffadc131200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.118+0000 7ffae18b1640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc12c500 0x7ffadc131740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.118+0000 7ffae18b1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffadc12ff10 0x7ffadc131c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.118+0000 7ffae18b1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc136970 con 0x7ffadc12ff10 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.118+0000 7ffae18b1640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7ffadc1367f0 con 0x7ffadc12c500 2024-09-15T21:57:00.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.118+0000 7ffae18b1640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7ffadc136af0 con 0x7ffadc09b600 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffada7fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc12c500 0x7ffadc131740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffadb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffadc12ff10 0x7ffadc131c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffadaffd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc09b600 0x7ffadc131200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffada7fc640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc12c500 0x7ffadc131740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45312/0 (socket says 172.21.15.47:45312) 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffadb7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffadc12ff10 0x7ffadc131c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57904/0 (socket says 172.21.15.47:57904) 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffada7fc640 1 -- 172.21.15.47:0/4198168852 learned_addr learned my addr 172.21.15.47:0/4198168852 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffada7fc640 1 -- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc09b600 msgr2=0x7ffadc131200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffada7fc640 1 --2- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc09b600 0x7ffadc131200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffada7fc640 1 -- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffadc12ff10 msgr2=0x7ffadc131c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffada7fc640 1 --2- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffadc12ff10 0x7ffadc131c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffada7fc640 1 -- 172.21.15.47:0/4198168852 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc134820 con 0x7ffadc12c500 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffadb7fe640 1 --2- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffadc12ff10 0x7ffadc131c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:00.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffada7fc640 1 --2- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc12c500 0x7ffadc131740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffad002a0e0 tx=0x7ffad00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffac3fff640 1 -- 172.21.15.47:0/4198168852 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad004c070 con 0x7ffadc12c500 2024-09-15T21:57:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffac3fff640 1 -- 172.21.15.47:0/4198168852 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad0032cd0 con 0x7ffadc12c500 2024-09-15T21:57:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffae18b1640 1 -- 172.21.15.47:0/4198168852 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffadc1333e0 con 0x7ffadc12c500 2024-09-15T21:57:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffac3fff640 1 -- 172.21.15.47:0/4198168852 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0037040 con 0x7ffadc12c500 2024-09-15T21:57:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.119+0000 7ffae18b1640 1 -- 172.21.15.47:0/4198168852 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffadc133f30 con 0x7ffadc12c500 2024-09-15T21:57:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.120+0000 7ffae18b1640 1 -- 172.21.15.47:0/4198168852 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffadc12c990 con 0x7ffadc12c500 2024-09-15T21:57:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.120+0000 7ffac3fff640 1 -- 172.21.15.47:0/4198168852 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7ffad004f070 con 0x7ffadc12c500 2024-09-15T21:57:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.121+0000 7ffac3fff640 1 -- 172.21.15.47:0/4198168852 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7ffad00859b0 con 0x7ffadc12c500 2024-09-15T21:57:00.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.124+0000 7ffac3fff640 1 -- 172.21.15.47:0/4198168852 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7ffad0044070 con 0x7ffadc12c500 2024-09-15T21:57:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.272+0000 7ffae18b1640 1 -- 172.21.15.47:0/4198168852 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffadc132c20 con 0x7ffac8047fd0 2024-09-15T21:57:00.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.273+0000 7ffac3fff640 1 -- 172.21.15.47:0/4198168852 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32004 (unknown 2965378022 0 1661938822) 0x7ffadc132c20 con 0x7ffac8047fd0 2024-09-15T21:57:00.275 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:57:00.276 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:57:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.274+0000 7ffae18b1640 1 -- 172.21.15.47:0/4198168852 >> v1:172.21.15.47:6800/1790418198 conn(0x7ffac8047fd0 legacy=0x7ffac804a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.274+0000 7ffae18b1640 1 -- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc12c500 msgr2=0x7ffadc131740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.274+0000 7ffae18b1640 1 --2- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc12c500 0x7ffadc131740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffad002a0e0 tx=0x7ffad00046c0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.274+0000 7ffae18b1640 1 -- 172.21.15.47:0/4198168852 shutdown_connections 2024-09-15T21:57:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.274+0000 7ffae18b1640 1 --2- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffadc12ff10 0x7ffadc131c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.274+0000 7ffae18b1640 1 --2- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7ffadc12c500 0x7ffadc131740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.274+0000 7ffae18b1640 1 --2- 172.21.15.47:0/4198168852 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7ffadc09b600 0x7ffadc131200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.275+0000 7ffae18b1640 1 -- 172.21.15.47:0/4198168852 >> 172.21.15.47:0/4198168852 conn(0x7ffadc09eb20 msgr2=0x7ffadc09d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.275+0000 7ffae18b1640 1 -- 172.21.15.47:0/4198168852 shutdown_connections 2024-09-15T21:57:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.275+0000 7ffae18b1640 1 -- 172.21.15.47:0/4198168852 wait complete. 2024-09-15T21:57:00.289 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-15T21:56:59.676524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128968,"kb_used_data":2192,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358392,"statfs":{"total":386547056640,"available":386414993408,"internally_reserved":0,"allocated":2244608,"data_stored":1543286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.943394"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185545+0000","last_change":"2024-09-15T21:56:49.850959+0000","last_active":"2024-09-15T21:56:56.185545+0000","last_peered":"2024-09-15T21:56:56.185545+0000","last_clean":"2024-09-15T21:56:56.185545+0000","last_became_active":"2024-09-15T21:56:49.850607+0000","last_became_peered":"2024-09-15T21:56:49.850607+0000","last_unstale":"2024-09-15T21:56:56.185545+0000","last_undegraded":"2024-09-15T21:56:56.185545+0000","last_fullsized":"2024-09-15T21:56:56.185545+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.532841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743750+0000","last_change":"2024-09-15T21:56:47.710370+0000","last_active":"2024-09-15T21:56:54.743750+0000","last_peered":"2024-09-15T21:56:54.743750+0000","last_clean":"2024-09-15T21:56:54.743750+0000","last_became_active":"2024-09-15T21:56:47.710067+0000","last_became_peered":"2024-09-15T21:56:47.710067+0000","last_unstale":"2024-09-15T21:56:54.743750+0000","last_undegraded":"2024-09-15T21:56:54.743750+0000","last_fullsized":"2024-09-15T21:56:54.743750+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:50.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185549+0000","last_change":"2024-09-15T21:56:49.851508+0000","last_active":"2024-09-15T21:56:56.185549+0000","last_peered":"2024-09-15T21:56:56.185549+0000","last_clean":"2024-09-15T21:56:56.185549+0000","last_became_active":"2024-09-15T21:56:49.851081+0000","last_became_peered":"2024-09-15T21:56:49.851081+0000","last_unstale":"2024-09-15T21:56:56.185549+0000","last_undegraded":"2024-09-15T21:56:56.185549+0000","last_fullsized":"2024-09-15T21:56:56.185549+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:03.698746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743740+0000","last_change":"2024-09-15T21:56:47.711356+0000","last_active":"2024-09-15T21:56:54.743740+0000","last_peered":"2024-09-15T21:56:54.743740+0000","last_clean":"2024-09-15T21:56:54.743740+0000","last_became_active":"2024-09-15T21:56:47.711012+0000","last_became_peered":"2024-09-15T21:56:47.711012+0000","last_unstale":"2024-09-15T21:56:54.743740+0000","last_undegraded":"2024-09-15T21:56:54.743740+0000","last_fullsized":"2024-09-15T21:56:54.743740+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:58:23.626385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185508+0000","last_change":"2024-09-15T21:56:49.733169+0000","last_active":"2024-09-15T21:56:56.185508+0000","last_peered":"2024-09-15T21:56:56.185508+0000","last_clean":"2024-09-15T21:56:56.185508+0000","last_became_active":"2024-09-15T21:56:49.732754+0000","last_became_peered":"2024-09-15T21:56:49.732754+0000","last_unstale":"2024-09-15T21:56:56.185508+0000","last_undegraded":"2024-09-15T21:56:56.185508+0000","last_fullsized":"2024-09-15T21:56:56.185508+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:11.761893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743733+0000","last_change":"2024-09-15T21:56:49.733734+0000","last_active":"2024-09-15T21:56:54.743733+0000","last_peered":"2024-09-15T21:56:54.743733+0000","last_clean":"2024-09-15T21:56:54.743733+0000","last_became_active":"2024-09-15T21:56:49.733153+0000","last_became_peered":"2024-09-15T21:56:49.733153+0000","last_unstale":"2024-09-15T21:56:54.743733+0000","last_undegraded":"2024-09-15T21:56:54.743733+0000","last_fullsized":"2024-09-15T21:56:54.743733+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:56.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185503+0000","last_change":"2024-09-15T21:56:49.731747+0000","last_active":"2024-09-15T21:56:56.185503+0000","last_peered":"2024-09-15T21:56:56.185503+0000","last_clean":"2024-09-15T21:56:56.185503+0000","last_became_active":"2024-09-15T21:56:49.731371+0000","last_became_peered":"2024-09-15T21:56:49.731371+0000","last_unstale":"2024-09-15T21:56:56.185503+0000","last_undegraded":"2024-09-15T21:56:56.185503+0000","last_fullsized":"2024-09-15T21:56:56.185503+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:27.373825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185514+0000","last_change":"2024-09-15T21:56:49.733242+0000","last_active":"2024-09-15T21:56:56.185514+0000","last_peered":"2024-09-15T21:56:56.185514+0000","last_clean":"2024-09-15T21:56:56.185514+0000","last_became_active":"2024-09-15T21:56:49.732855+0000","last_became_peered":"2024-09-15T21:56:49.732855+0000","last_unstale":"2024-09-15T21:56:56.185514+0000","last_undegraded":"2024-09-15T21:56:56.185514+0000","last_fullsized":"2024-09-15T21:56:56.185514+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:50:43.371057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":146,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743991+0000","last_change":"2024-09-15T21:56:47.710563+0000","last_active":"2024-09-15T21:56:54.743991+0000","last_peered":"2024-09-15T21:56:54.743991+0000","last_clean":"2024-09-15T21:56:54.743991+0000","last_became_active":"2024-09-15T21:56:47.710235+0000","last_became_peered":"2024-09-15T21:56:47.710235+0000","last_unstale":"2024-09-15T21:56:54.743991+0000","last_undegraded":"2024-09-15T21:56:54.743991+0000","last_fullsized":"2024-09-15T21:56:54.743991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_clean_scrub_stamp":"2024-09-15T21:55:11.249936+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:43.977752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":27,"seq":115964116996,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":266240,"data_stored":90647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":27,"seq":115964116996,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":262144,"data_stored":90628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":840,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":860160,"data_stored":681015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":836,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":856064,"data_stored":680996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:57:00.289 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:57:00.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:57:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.390+0000 7f1521aaa640 1 Processor -- start 2024-09-15T21:57:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.390+0000 7f1521aaa640 1 -- start start 2024-09-15T21:57:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.390+0000 7f1521aaa640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f151c12f3c0 0x7f151c12f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.390+0000 7f1521aaa640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c098090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.390+0000 7f1521aaa640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f151c09f230 con 0x7f151c0985d0 2024-09-15T21:57:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.390+0000 7f1521aaa640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f151c06f260 con 0x7f151c12f3c0 2024-09-15T21:57:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.390+0000 7f1521aaa640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f151c099600 con 0x7f151c12fce0 2024-09-15T21:57:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151affd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c098090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151b7fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f151c12f3c0 0x7f151c12f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151bfff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f151c0985d0 0x7f151c136920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34222/0 (socket says 172.21.15.47:34222) 2024-09-15T21:57:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151affd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c098090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45328/0 (socket says 172.21.15.47:45328) 2024-09-15T21:57:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151bfff640 1 -- 172.21.15.47:0/3003529779 learned_addr learned my addr 172.21.15.47:0/3003529779 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:57:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151a7fc640 1 -- 172.21.15.47:0/3003529779 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4260510883 0 0) 0x7f151c09f230 con 0x7f151c0985d0 2024-09-15T21:57:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151affd640 1 -- 172.21.15.47:0/3003529779 >> v2:172.21.15.47:3300/0 conn(0x7f151c12f3c0 msgr2=0x7f151c12f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151affd640 1 --2- 172.21.15.47:0/3003529779 >> v2:172.21.15.47:3300/0 conn(0x7f151c12f3c0 0x7f151c12f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151affd640 1 -- 172.21.15.47:0/3003529779 >> v1:172.21.15.47:6789/0 conn(0x7f151c0985d0 legacy=0x7f151c136920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151affd640 1 -- 172.21.15.47:0/3003529779 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c130120 con 0x7f151c12fce0 2024-09-15T21:57:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.391+0000 7f151b7fe640 1 --2- 172.21.15.47:0/3003529779 >> v2:172.21.15.47:3300/0 conn(0x7f151c12f3c0 0x7f151c12f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f151affd640 1 --2- 172.21.15.47:0/3003529779 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c098090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f151c099780 tx=0x7f150c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89707a2fc90bf8c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f151a7fc640 1 -- 172.21.15.47:0/3003529779 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c037070 con 0x7f151c12fce0 2024-09-15T21:57:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f151a7fc640 1 -- 172.21.15.47:0/3003529779 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c02adb0 con 0x7f151c12fce0 2024-09-15T21:57:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f151a7fc640 1 -- 172.21.15.47:0/3003529779 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c032660 con 0x7f151c12fce0 2024-09-15T21:57:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f1521aaa640 1 -- 172.21.15.47:0/3003529779 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 msgr2=0x7f151c098090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f1521aaa640 1 --2- 172.21.15.47:0/3003529779 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c098090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f151c099780 tx=0x7f150c02a060 comp rx=0 tx=0).stop 2024-09-15T21:57:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f1521aaa640 1 -- 172.21.15.47:0/3003529779 shutdown_connections 2024-09-15T21:57:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f1521aaa640 1 --2- 172.21.15.47:0/3003529779 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c098090 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f1521aaa640 1 --2- 172.21.15.47:0/3003529779 >> v2:172.21.15.47:3300/0 conn(0x7f151c12f3c0 0x7f151c12f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f1521aaa640 1 -- 172.21.15.47:0/3003529779 >> 172.21.15.47:0/3003529779 conn(0x7f151c0a23b0 msgr2=0x7f151c09e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f1521aaa640 1 -- 172.21.15.47:0/3003529779 shutdown_connections 2024-09-15T21:57:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.392+0000 7f1521aaa640 1 -- 172.21.15.47:0/3003529779 wait complete. 2024-09-15T21:57:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.393+0000 7f1521aaa640 1 Processor -- start 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f1521aaa640 1 -- start start 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f1521aaa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0985d0 0x7f151c1c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f1521aaa640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f151c12f3c0 0x7f151c1c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f1521aaa640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c08cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f1521aaa640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f151c098a40 con 0x7f151c0985d0 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f1521aaa640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f151c06ee60 con 0x7f151c12fce0 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f1521aaa640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f151c098bc0 con 0x7f151c12f3c0 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f151b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0985d0 0x7f151c1c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f151affd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f151c12f3c0 0x7f151c1c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f151b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0985d0 0x7f151c1c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57922/0 (socket says 172.21.15.47:57922) 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f151b7fe640 1 -- 172.21.15.47:0/3972930680 learned_addr learned my addr 172.21.15.47:0/3972930680 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.394+0000 7f151bfff640 1 --2- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c08cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f151b7fe640 1 -- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f151c12f3c0 msgr2=0x7f151c1c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f151b7fe640 1 --2- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f151c12f3c0 0x7f151c1c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f151b7fe640 1 -- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 msgr2=0x7f151c08cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f151b7fe640 1 --2- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c08cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f151b7fe640 1 -- 172.21.15.47:0/3972930680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c08e850 con 0x7f151c0985d0 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f151bfff640 1 --2- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c08cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f151affd640 1 --2- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f151c12f3c0 0x7f151c1c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f151b7fe640 1 --2- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0985d0 0x7f151c1c4960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f150400ebe0 tx=0x7f150400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f1513fff640 1 -- 172.21.15.47:0/3972930680 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504017070 con 0x7f151c0985d0 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f1513fff640 1 -- 172.21.15.47:0/3972930680 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150400cc30 con 0x7f151c0985d0 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f1521aaa640 1 -- 172.21.15.47:0/3972930680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c1c53e0 con 0x7f151c0985d0 2024-09-15T21:57:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f1513fff640 1 -- 172.21.15.47:0/3972930680 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504013470 con 0x7f151c0985d0 2024-09-15T21:57:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.395+0000 7f1521aaa640 1 -- 172.21.15.47:0/3972930680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c08faf0 con 0x7f151c0985d0 2024-09-15T21:57:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.396+0000 7f1513fff640 1 -- 172.21.15.47:0/3972930680 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f150401a070 con 0x7f151c0985d0 2024-09-15T21:57:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.396+0000 7f1521aaa640 1 -- 172.21.15.47:0/3972930680 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151c12f7a0 con 0x7f151c0985d0 2024-09-15T21:57:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.397+0000 7f1513fff640 1 -- 172.21.15.47:0/3972930680 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f150405d970 con 0x7f151c0985d0 2024-09-15T21:57:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.400+0000 7f1513fff640 1 -- 172.21.15.47:0/3972930680 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f150403cac0 con 0x7f151c0985d0 2024-09-15T21:57:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.547+0000 7f1521aaa640 1 -- 172.21.15.47:0/3972930680 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f151c0983b0 con 0x7f14fc047fd0 2024-09-15T21:57:00.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.548+0000 7f1513fff640 1 -- 172.21.15.47:0/3972930680 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32004 (unknown 2965378022 0 1661938822) 0x7f151c0983b0 con 0x7f14fc047fd0 2024-09-15T21:57:00.551 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:57:00.551 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:57:00.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.550+0000 7f1521aaa640 1 -- 172.21.15.47:0/3972930680 >> v1:172.21.15.47:6800/1790418198 conn(0x7f14fc047fd0 legacy=0x7f14fc04a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.550+0000 7f1521aaa640 1 -- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0985d0 msgr2=0x7f151c1c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.550+0000 7f1521aaa640 1 --2- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0985d0 0x7f151c1c4960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f150400ebe0 tx=0x7f150400c520 comp rx=0 tx=0).stop 2024-09-15T21:57:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.550+0000 7f1521aaa640 1 -- 172.21.15.47:0/3972930680 shutdown_connections 2024-09-15T21:57:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.550+0000 7f1521aaa640 1 --2- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f151c12fce0 0x7f151c08cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.550+0000 7f1521aaa640 1 --2- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f151c12f3c0 0x7f151c1c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.550+0000 7f1521aaa640 1 --2- 172.21.15.47:0/3972930680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f151c0985d0 0x7f151c1c4960 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.550+0000 7f1521aaa640 1 -- 172.21.15.47:0/3972930680 >> 172.21.15.47:0/3972930680 conn(0x7f151c0a23b0 msgr2=0x7f151c099b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.550+0000 7f1521aaa640 1 -- 172.21.15.47:0/3972930680 shutdown_connections 2024-09-15T21:57:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.550+0000 7f1521aaa640 1 -- 172.21.15.47:0/3972930680 wait complete. 2024-09-15T21:57:00.564 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-15T21:56:59.676524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128968,"kb_used_data":2192,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358392,"statfs":{"total":386547056640,"available":386414993408,"internally_reserved":0,"allocated":2244608,"data_stored":1543286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.943394"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185545+0000","last_change":"2024-09-15T21:56:49.850959+0000","last_active":"2024-09-15T21:56:56.185545+0000","last_peered":"2024-09-15T21:56:56.185545+0000","last_clean":"2024-09-15T21:56:56.185545+0000","last_became_active":"2024-09-15T21:56:49.850607+0000","last_became_peered":"2024-09-15T21:56:49.850607+0000","last_unstale":"2024-09-15T21:56:56.185545+0000","last_undegraded":"2024-09-15T21:56:56.185545+0000","last_fullsized":"2024-09-15T21:56:56.185545+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.532841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743750+0000","last_change":"2024-09-15T21:56:47.710370+0000","last_active":"2024-09-15T21:56:54.743750+0000","last_peered":"2024-09-15T21:56:54.743750+0000","last_clean":"2024-09-15T21:56:54.743750+0000","last_became_active":"2024-09-15T21:56:47.710067+0000","last_became_peered":"2024-09-15T21:56:47.710067+0000","last_unstale":"2024-09-15T21:56:54.743750+0000","last_undegraded":"2024-09-15T21:56:54.743750+0000","last_fullsized":"2024-09-15T21:56:54.743750+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:50.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185549+0000","last_change":"2024-09-15T21:56:49.851508+0000","last_active":"2024-09-15T21:56:56.185549+0000","last_peered":"2024-09-15T21:56:56.185549+0000","last_clean":"2024-09-15T21:56:56.185549+0000","last_became_active":"2024-09-15T21:56:49.851081+0000","last_became_peered":"2024-09-15T21:56:49.851081+0000","last_unstale":"2024-09-15T21:56:56.185549+0000","last_undegraded":"2024-09-15T21:56:56.185549+0000","last_fullsized":"2024-09-15T21:56:56.185549+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:03.698746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743740+0000","last_change":"2024-09-15T21:56:47.711356+0000","last_active":"2024-09-15T21:56:54.743740+0000","last_peered":"2024-09-15T21:56:54.743740+0000","last_clean":"2024-09-15T21:56:54.743740+0000","last_became_active":"2024-09-15T21:56:47.711012+0000","last_became_peered":"2024-09-15T21:56:47.711012+0000","last_unstale":"2024-09-15T21:56:54.743740+0000","last_undegraded":"2024-09-15T21:56:54.743740+0000","last_fullsized":"2024-09-15T21:56:54.743740+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:58:23.626385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185508+0000","last_change":"2024-09-15T21:56:49.733169+0000","last_active":"2024-09-15T21:56:56.185508+0000","last_peered":"2024-09-15T21:56:56.185508+0000","last_clean":"2024-09-15T21:56:56.185508+0000","last_became_active":"2024-09-15T21:56:49.732754+0000","last_became_peered":"2024-09-15T21:56:49.732754+0000","last_unstale":"2024-09-15T21:56:56.185508+0000","last_undegraded":"2024-09-15T21:56:56.185508+0000","last_fullsized":"2024-09-15T21:56:56.185508+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:11.761893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743733+0000","last_change":"2024-09-15T21:56:49.733734+0000","last_active":"2024-09-15T21:56:54.743733+0000","last_peered":"2024-09-15T21:56:54.743733+0000","last_clean":"2024-09-15T21:56:54.743733+0000","last_became_active":"2024-09-15T21:56:49.733153+0000","last_became_peered":"2024-09-15T21:56:49.733153+0000","last_unstale":"2024-09-15T21:56:54.743733+0000","last_undegraded":"2024-09-15T21:56:54.743733+0000","last_fullsized":"2024-09-15T21:56:54.743733+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:56.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185503+0000","last_change":"2024-09-15T21:56:49.731747+0000","last_active":"2024-09-15T21:56:56.185503+0000","last_peered":"2024-09-15T21:56:56.185503+0000","last_clean":"2024-09-15T21:56:56.185503+0000","last_became_active":"2024-09-15T21:56:49.731371+0000","last_became_peered":"2024-09-15T21:56:49.731371+0000","last_unstale":"2024-09-15T21:56:56.185503+0000","last_undegraded":"2024-09-15T21:56:56.185503+0000","last_fullsized":"2024-09-15T21:56:56.185503+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:27.373825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185514+0000","last_change":"2024-09-15T21:56:49.733242+0000","last_active":"2024-09-15T21:56:56.185514+0000","last_peered":"2024-09-15T21:56:56.185514+0000","last_clean":"2024-09-15T21:56:56.185514+0000","last_became_active":"2024-09-15T21:56:49.732855+0000","last_became_peered":"2024-09-15T21:56:49.732855+0000","last_unstale":"2024-09-15T21:56:56.185514+0000","last_undegraded":"2024-09-15T21:56:56.185514+0000","last_fullsized":"2024-09-15T21:56:56.185514+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:50:43.371057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":146,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743991+0000","last_change":"2024-09-15T21:56:47.710563+0000","last_active":"2024-09-15T21:56:54.743991+0000","last_peered":"2024-09-15T21:56:54.743991+0000","last_clean":"2024-09-15T21:56:54.743991+0000","last_became_active":"2024-09-15T21:56:47.710235+0000","last_became_peered":"2024-09-15T21:56:47.710235+0000","last_unstale":"2024-09-15T21:56:54.743991+0000","last_undegraded":"2024-09-15T21:56:54.743991+0000","last_fullsized":"2024-09-15T21:56:54.743991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_clean_scrub_stamp":"2024-09-15T21:55:11.249936+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:43.977752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":27,"seq":115964116996,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":266240,"data_stored":90647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":27,"seq":115964116996,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":262144,"data_stored":90628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":840,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":860160,"data_stored":681015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":836,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":856064,"data_stored":680996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:57:00.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:57:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.664+0000 7f7eaf02b640 1 Processor -- start 2024-09-15T21:57:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.664+0000 7f7eaf02b640 1 -- start start 2024-09-15T21:57:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.664+0000 7f7eaf02b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8072e40 0x7f7ea8073220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7eaf02b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 0x7f7ea812de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7eaf02b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea8072670 con 0x7f7ea8073760 2024-09-15T21:57:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7eaf02b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea8071560 con 0x7f7ea8072e40 2024-09-15T21:57:00.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7eaf02b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f7ea8071e40 con 0x7f7ea812d9d0 2024-09-15T21:57:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7eacdc7640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8072e40 0x7f7ea8073220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7e9ffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7ea8073760 0x7f7ea812d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34226/0 (socket says 172.21.15.47:34226) 2024-09-15T21:57:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7ead5c8640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 0x7f7ea812de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7e9ffff640 1 -- 172.21.15.47:0/1171217813 learned_addr learned my addr 172.21.15.47:0/1171217813 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:57:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7eacdc7640 1 -- 172.21.15.47:0/1171217813 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 msgr2=0x7f7ea812de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/1171217813 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4068972851 0 0) 0x7f7ea8072670 con 0x7f7ea8073760 2024-09-15T21:57:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7eacdc7640 1 --2- 172.21.15.47:0/1171217813 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 0x7f7ea812de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7eacdc7640 1 -- 172.21.15.47:0/1171217813 >> v1:172.21.15.47:6789/0 conn(0x7f7ea8073760 legacy=0x7f7ea812d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.665+0000 7f7eacdc7640 1 -- 172.21.15.47:0/1171217813 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea8072910 con 0x7f7ea8072e40 2024-09-15T21:57:00.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.666+0000 7f7eacdc7640 1 --2- 172.21.15.47:0/1171217813 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8072e40 0x7f7ea8073220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40097d0 tx=0x7f7ea402a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d5ac8edd13728c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:00.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.666+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/1171217813 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4037070 con 0x7f7ea8072e40 2024-09-15T21:57:00.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.666+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/1171217813 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea402ad90 con 0x7f7ea8072e40 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.666+0000 7f7e9f7fe640 1 -- 172.21.15.47:0/1171217813 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4032660 con 0x7f7ea8072e40 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.667+0000 7f7eaf02b640 1 -- 172.21.15.47:0/1171217813 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8072e40 msgr2=0x7f7ea8073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.667+0000 7f7eaf02b640 1 --2- 172.21.15.47:0/1171217813 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8072e40 0x7f7ea8073220 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40097d0 tx=0x7f7ea402a040 comp rx=0 tx=0).stop 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.667+0000 7f7eaf02b640 1 -- 172.21.15.47:0/1171217813 shutdown_connections 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.667+0000 7f7eaf02b640 1 --2- 172.21.15.47:0/1171217813 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 0x7f7ea812de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.667+0000 7f7eaf02b640 1 --2- 172.21.15.47:0/1171217813 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8072e40 0x7f7ea8073220 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.667+0000 7f7eaf02b640 1 -- 172.21.15.47:0/1171217813 >> 172.21.15.47:0/1171217813 conn(0x7f7ea809fcf0 msgr2=0x7f7ea80a0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.667+0000 7f7eaf02b640 1 -- 172.21.15.47:0/1171217813 shutdown_connections 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.667+0000 7f7eaf02b640 1 -- 172.21.15.47:0/1171217813 wait complete. 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.668+0000 7f7eaf02b640 1 Processor -- start 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.668+0000 7f7eaf02b640 1 -- start start 2024-09-15T21:57:00.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.668+0000 7f7eaf02b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea8072e40 0x7f7ea812f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.668+0000 7f7eaf02b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8073760 0x7f7ea812fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.668+0000 7f7eaf02b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 0x7f7ea81303c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.668+0000 7f7eaf02b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8135780 con 0x7f7ea8072e40 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.668+0000 7f7eaf02b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea8135600 con 0x7f7ea8073760 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.668+0000 7f7eaf02b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f7ea8135900 con 0x7f7ea812d9d0 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eacdc7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea8072e40 0x7f7ea812f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7ead5c8640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 0x7f7ea81303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eacdc7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea8072e40 0x7f7ea812f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57938/0 (socket says 172.21.15.47:57938) 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eacdc7640 1 -- 172.21.15.47:0/3046839593 learned_addr learned my addr 172.21.15.47:0/3046839593 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7e9ffff640 1 --2- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8073760 0x7f7ea812fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:00.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eacdc7640 1 -- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 msgr2=0x7f7ea81303c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eacdc7640 1 --2- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 0x7f7ea81303c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eacdc7640 1 -- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8073760 msgr2=0x7f7ea812fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eacdc7640 1 --2- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8073760 0x7f7ea812fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eacdc7640 1 -- 172.21.15.47:0/3046839593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea8131c40 con 0x7f7ea8072e40 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7ead5c8640 1 --2- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 0x7f7ea81303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eacdc7640 1 --2- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea8072e40 0x7f7ea812f940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40098f0 tx=0x7f7ea40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7e9d7fa640 1 -- 172.21.15.47:0/3046839593 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4041020 con 0x7f7ea8072e40 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7e9d7fa640 1 -- 172.21.15.47:0/3046839593 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea403d420 con 0x7f7ea8072e40 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eaf02b640 1 -- 172.21.15.47:0/3046839593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea8131360 con 0x7f7ea8072e40 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7e9d7fa640 1 -- 172.21.15.47:0/3046839593 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4037040 con 0x7f7ea8072e40 2024-09-15T21:57:00.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.669+0000 7f7eaf02b640 1 -- 172.21.15.47:0/3046839593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea81332b0 con 0x7f7ea8072e40 2024-09-15T21:57:00.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.670+0000 7f7eaf02b640 1 -- 172.21.15.47:0/3046839593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea80a0210 con 0x7f7ea8072e40 2024-09-15T21:57:00.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.671+0000 7f7e9d7fa640 1 -- 172.21.15.47:0/3046839593 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f7ea4044020 con 0x7f7ea8072e40 2024-09-15T21:57:00.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.671+0000 7f7e9d7fa640 1 -- 172.21.15.47:0/3046839593 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f7ea407db60 con 0x7f7ea8072e40 2024-09-15T21:57:00.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.677+0000 7f7e9d7fa640 1 -- 172.21.15.47:0/3046839593 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f7ea80a0210 con 0x7f7ea8072e40 2024-09-15T21:57:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.876+0000 7f7eaf02b640 1 -- 172.21.15.47:0/3046839593 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ea809add0 con 0x7f7ea8072e40 2024-09-15T21:57:00.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.876+0000 7f7e9d7fa640 1 -- 172.21.15.47:0/3046839593 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+9172 (secure 0 0 0) 0x7f7ea809add0 con 0x7f7ea8072e40 2024-09-15T21:57:00.879 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:57:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":31,"fsid":"519e4c2c-d517-4539-a86b-3d427c67bae0","created":"2024-09-15T21:55:00.295766+0000","modified":"2024-09-15T21:56:54.740172+0000","last_up_change":"2024-09-15T21:56:48.715593+0000","last_in_change":"2024-09-15T21:55:02.238284+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:55:10.740066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:55:16.666135+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3f514164-0bf2-4d96-99c2-dc948a0d3cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6801","nonce":2079261393}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6802","nonce":2079261393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6804","nonce":2079261393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6803","nonce":2079261393}]},"public_addr":"172.21.15.47:6801/2079261393","cluster_addr":"172.21.15.47:6802/2079261393","heartbeat_back_addr":"172.21.15.47:6804/2079261393","heartbeat_front_addr":"172.21.15.47:6803/2079261393","state":["exists","up"]},{"osd":1,"uuid":"98cbb40c-7873-4bb4-8e27-42dd62549685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":25,"up_thru":25,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6805","nonce":4287109391}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6806","nonce":4287109391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6808","nonce":4287109391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6807","nonce":4287109391}]},"public_addr":"172.21.15.47:6805/4287109391","cluster_addr":"172.21.15.47:6806/4287109391","heartbeat_back_addr":"172.21.15.47:6808/4287109391","heartbeat_front_addr":"172.21.15.47:6807/4287109391","state":["exists","up"]},{"osd":2,"uuid":"cd82ae94-f43c-476f-bb97-a0699b859536","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":27,"up_thru":27,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6809","nonce":2903715449}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6810","nonce":2903715449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6812","nonce":2903715449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6811","nonce":2903715449}]},"public_addr":"172.21.15.47:6809/2903715449","cluster_addr":"172.21.15.47:6810/2903715449","heartbeat_back_addr":"172.21.15.47:6812/2903715449","heartbeat_front_addr":"172.21.15.47:6811/2903715449","state":["exists","up"]},{"osd":3,"uuid":"708e9506-4b05-4c61-95fa-a920cdc9d8fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":27,"up_thru":27,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6813","nonce":2978958959}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6814","nonce":2978958959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6816","nonce":2978958959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.47:6815","nonce":2978958959}]},"public_addr":"172.21.15.47:6813/2978958959","cluster_addr":"172.21.15.47:6814/2978958959","heartbeat_back_addr":"172.21.15.47:6816/2978958959","heartbeat_front_addr":"172.21.15.47:6815/2978958959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.272012+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.321013+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.397110+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-15T21:56:40.690629+0000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:55:07.350708+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:0/3200219561":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/2444109955":"2024-09-16T21:56:33.661886+0000","172.21.15.47:6800/32267":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1739125583":"2024-09-16T21:56:33.661886+0000","172.21.15.47:0/1981014788":"2024-09-16T21:56:33.661886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:57:00.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.878+0000 7f7eaf02b640 1 -- 172.21.15.47:0/3046839593 >> v1:172.21.15.47:6800/1790418198 conn(0x7f7e94047fd0 legacy=0x7f7e9404a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.878+0000 7f7eaf02b640 1 -- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea8072e40 msgr2=0x7f7ea812f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:00.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.878+0000 7f7eaf02b640 1 --2- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea8072e40 0x7f7ea812f940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40098f0 tx=0x7f7ea40046c0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.878+0000 7f7eaf02b640 1 -- 172.21.15.47:0/3046839593 shutdown_connections 2024-09-15T21:57:00.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.878+0000 7f7eaf02b640 1 --2- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7ea812d9d0 0x7f7ea81303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.878+0000 7f7eaf02b640 1 --2- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7ea8073760 0x7f7ea812fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.878+0000 7f7eaf02b640 1 --2- 172.21.15.47:0/3046839593 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7ea8072e40 0x7f7ea812f940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:00.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.878+0000 7f7eaf02b640 1 -- 172.21.15.47:0/3046839593 >> 172.21.15.47:0/3046839593 conn(0x7f7ea809fcf0 msgr2=0x7f7ea8126be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:00.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.878+0000 7f7eaf02b640 1 -- 172.21.15.47:0/3046839593 shutdown_connections 2024-09-15T21:57:00.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:00.878+0000 7f7eaf02b640 1 -- 172.21.15.47:0/3046839593 wait complete. 2024-09-15T21:57:01.893 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T21:57:01.893 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:57:02.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.003+0000 7fe34a06b640 1 Processor -- start 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.003+0000 7fe34a06b640 1 -- start start 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.003+0000 7fe34a06b640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe34409ba50 0x7fe34406de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.003+0000 7fe34a06b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 0x7fe344137110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.003+0000 7fe34a06b640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe344070150 con 0x7fe34406f970 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.003+0000 7fe34a06b640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe34406f370 con 0x7fe34409ba50 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.003+0000 7fe34a06b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fe34409c3b0 con 0x7fe344134d20 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.004+0000 7fe343fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 0x7fe344137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.004+0000 7fe3437fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe34409ba50 0x7fe34406de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.004+0000 7fe343fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 0x7fe344137110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45382/0 (socket says 172.21.15.47:45382) 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.004+0000 7fe343fff640 1 -- 172.21.15.47:0/3992884545 learned_addr learned my addr 172.21.15.47:0/3992884545 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.004+0000 7fe343fff640 1 -- 172.21.15.47:0/3992884545 >> v2:172.21.15.47:3300/0 conn(0x7fe34409ba50 msgr2=0x7fe34406de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.004+0000 7fe343fff640 1 --2- 172.21.15.47:0/3992884545 >> v2:172.21.15.47:3300/0 conn(0x7fe34409ba50 0x7fe34406de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.004+0000 7fe343fff640 1 -- 172.21.15.47:0/3992884545 >> v1:172.21.15.47:6789/0 conn(0x7fe34406f970 legacy=0x7fe34406e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.004+0000 7fe3427fc640 1 -- 172.21.15.47:0/3992884545 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962923085 0 0) 0x7fe344070150 con 0x7fe34406f970 2024-09-15T21:57:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.004+0000 7fe343fff640 1 -- 172.21.15.47:0/3992884545 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34406ea90 con 0x7fe344134d20 2024-09-15T21:57:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.004+0000 7fe3437fe640 1 --2- 172.21.15.47:0/3992884545 >> v2:172.21.15.47:3300/0 conn(0x7fe34409ba50 0x7fe34406de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe343fff640 1 --2- 172.21.15.47:0/3992884545 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 0x7fe344137110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009e80 tx=0x7fe33c02a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9dc9182f83bdae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe3427fc640 1 -- 172.21.15.47:0/3992884545 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c036070 con 0x7fe344134d20 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe3427fc640 1 -- 172.21.15.47:0/3992884545 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe33c031600 con 0x7fe344134d20 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe3427fc640 1 -- 172.21.15.47:0/3992884545 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0319d0 con 0x7fe344134d20 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe34a06b640 1 -- 172.21.15.47:0/3992884545 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 msgr2=0x7fe344137110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe34a06b640 1 --2- 172.21.15.47:0/3992884545 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 0x7fe344137110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009e80 tx=0x7fe33c02a630 comp rx=0 tx=0).stop 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe34a06b640 1 -- 172.21.15.47:0/3992884545 shutdown_connections 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe34a06b640 1 --2- 172.21.15.47:0/3992884545 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 0x7fe344137110 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe34a06b640 1 --2- 172.21.15.47:0/3992884545 >> v2:172.21.15.47:3300/0 conn(0x7fe34409ba50 0x7fe34406de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe34a06b640 1 -- 172.21.15.47:0/3992884545 >> 172.21.15.47:0/3992884545 conn(0x7fe34409caa0 msgr2=0x7fe34409ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe34a06b640 1 -- 172.21.15.47:0/3992884545 shutdown_connections 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.005+0000 7fe34a06b640 1 -- 172.21.15.47:0/3992884545 wait complete. 2024-09-15T21:57:02.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.006+0000 7fe34a06b640 1 Processor -- start 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe34a06b640 1 -- start start 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe34a06b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe34406f970 0x7fe344133400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe34a06b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe34409ba50 0x7fe344133940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe34a06b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 0x7fe344133e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe34a06b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441379e0 con 0x7fe34406f970 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe34a06b640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fe344137860 con 0x7fe344134d20 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe34a06b640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fe344137b60 con 0x7fe34409ba50 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe3437fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe34406f970 0x7fe344133400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe343fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 0x7fe344133e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe3437fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe34406f970 0x7fe344133400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57956/0 (socket says 172.21.15.47:57956) 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe3437fe640 1 -- 172.21.15.47:0/1815649503 learned_addr learned my addr 172.21.15.47:0/1815649503 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe342ffd640 1 --2- 172.21.15.47:0/1815649503 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe34409ba50 0x7fe344133940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe3437fe640 1 -- 172.21.15.47:0/1815649503 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe34409ba50 msgr2=0x7fe344133940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe3437fe640 1 --2- 172.21.15.47:0/1815649503 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fe34409ba50 0x7fe344133940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe3437fe640 1 -- 172.21.15.47:0/1815649503 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 msgr2=0x7fe344133e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe3437fe640 1 --2- 172.21.15.47:0/1815649503 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 0x7fe344133e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe3437fe640 1 -- 172.21.15.47:0/1815649503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe344091550 con 0x7fe34406f970 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.007+0000 7fe343fff640 1 --2- 172.21.15.47:0/1815649503 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fe344134d20 0x7fe344133e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.008+0000 7fe3437fe640 1 --2- 172.21.15.47:0/1815649503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe34406f970 0x7fe344133400 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe33400ebe0 tx=0x7fe33400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.008+0000 7fe33bfff640 1 -- 172.21.15.47:0/1815649503 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334017070 con 0x7fe34406f970 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.008+0000 7fe33bfff640 1 -- 172.21.15.47:0/1815649503 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe33400cbd0 con 0x7fe34406f970 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.008+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3440921c0 con 0x7fe34406f970 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.008+0000 7fe33bfff640 1 -- 172.21.15.47:0/1815649503 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334013470 con 0x7fe34406f970 2024-09-15T21:57:02.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.008+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe344099200 con 0x7fe34406f970 2024-09-15T21:57:02.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.009+0000 7fe33bfff640 1 -- 172.21.15.47:0/1815649503 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fe33401a070 con 0x7fe34406f970 2024-09-15T21:57:02.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.009+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe34406de40 con 0x7fe34406f970 2024-09-15T21:57:02.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.009+0000 7fe33bfff640 1 -- 172.21.15.47:0/1815649503 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fe33405d8b0 con 0x7fe34406f970 2024-09-15T21:57:02.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.009+0000 7fe33bfff640 1 -- 172.21.15.47:0/1815649503 --> v1:172.21.15.47:6801/2079261393 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe334016ec0 con 0x7fe32404e850 2024-09-15T21:57:02.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.009+0000 7fe33bfff640 1 -- 172.21.15.47:0/1815649503 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe3340622c0 con 0x7fe34406f970 2024-09-15T21:57:02.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.010+0000 7fe33bfff640 1 -- 172.21.15.47:0/1815649503 <== osd.0 v1:172.21.15.47:6801/2079261393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fe334016ec0 con 0x7fe32404e850 2024-09-15T21:57:02.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 --> v1:172.21.15.47:6801/2079261393 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe344092900 con 0x7fe32404e850 2024-09-15T21:57:02.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe33bfff640 1 -- 172.21.15.47:0/1815649503 <== osd.0 v1:172.21.15.47:6801/2079261393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (unknown 0 0 125856537) 0x7fe344092900 con 0x7fe32404e850 2024-09-15T21:57:02.025 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T21:57:02.026 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-15T21:57:02.026 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T21:57:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 >> v1:172.21.15.47:6801/2079261393 conn(0x7fe32404e850 legacy=0x7fe324050c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 >> v1:172.21.15.47:6800/1790418198 conn(0x7fe324047fd0 legacy=0x7fe32404a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe34406f970 msgr2=0x7fe344133400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe34a06b640 1 --2- 172.21.15.47:0/1815649503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe34406f970 0x7fe344133400 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe33400ebe0 tx=0x7fe33400c520 comp rx=0 tx=0).stop 2024-09-15T21:57:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe343fff640 1 -- 172.21.15.47:0/1815649503 reap_dead start 2024-09-15T21:57:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 shutdown_connections 2024-09-15T21:57:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 >> 172.21.15.47:0/1815649503 conn(0x7fe34409caa0 msgr2=0x7fe34412fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 shutdown_connections 2024-09-15T21:57:02.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.023+0000 7fe34a06b640 1 -- 172.21.15.47:0/1815649503 wait complete. 2024-09-15T21:57:02.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T21:57:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a3678640 1 Processor -- start 2024-09-15T21:57:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a3678640 1 -- start start 2024-09-15T21:57:02.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a3678640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c096b00 0x7f269c096ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a3678640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f269c094760 0x7f269c134760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a3678640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f269c06cfd0 con 0x7f269c097420 2024-09-15T21:57:02.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a3678640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f269c06e1a0 con 0x7f269c094760 2024-09-15T21:57:02.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a3678640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f269c06ccf0 con 0x7f269c096b00 2024-09-15T21:57:02.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a1414640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c096b00 0x7f269c096ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a1c15640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f269c094760 0x7f269c134760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a1414640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c096b00 0x7f269c096ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36976/0 (socket says 172.21.15.47:36976) 2024-09-15T21:57:02.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a0c13640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f269c097420 0x7f269c094050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34240/0 (socket says 172.21.15.47:34240) 2024-09-15T21:57:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.136+0000 7f26a1414640 1 -- 172.21.15.47:0/53836454 learned_addr learned my addr 172.21.15.47:0/53836454 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.137+0000 7f26a1414640 1 -- 172.21.15.47:0/53836454 >> v2:172.21.15.47:3300/0 conn(0x7f269c094760 msgr2=0x7f269c134760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.137+0000 7f26a1414640 1 --2- 172.21.15.47:0/53836454 >> v2:172.21.15.47:3300/0 conn(0x7f269c094760 0x7f269c134760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.137+0000 7f26a1414640 1 -- 172.21.15.47:0/53836454 >> v1:172.21.15.47:6789/0 conn(0x7f269c097420 legacy=0x7f269c094050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.137+0000 7f26a1414640 1 -- 172.21.15.47:0/53836454 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c06de90 con 0x7f269c096b00 2024-09-15T21:57:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.137+0000 7f26a1c15640 1 --2- 172.21.15.47:0/53836454 >> v2:172.21.15.47:3300/0 conn(0x7f269c094760 0x7f269c134760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.137+0000 7f26a1414640 1 --2- 172.21.15.47:0/53836454 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c096b00 0x7f269c096ee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f269800bb10 tx=0x7f269802c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=951afc49e8e92fe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f2693fff640 1 -- 172.21.15.47:0/53836454 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698032700 con 0x7f269c096b00 2024-09-15T21:57:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f2693fff640 1 -- 172.21.15.47:0/53836454 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26980328a0 con 0x7f269c096b00 2024-09-15T21:57:02.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f2693fff640 1 -- 172.21.15.47:0/53836454 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698032c70 con 0x7f269c096b00 2024-09-15T21:57:02.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f26a3678640 1 -- 172.21.15.47:0/53836454 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c096b00 msgr2=0x7f269c096ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f26a3678640 1 --2- 172.21.15.47:0/53836454 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c096b00 0x7f269c096ee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f269800bb10 tx=0x7f269802c800 comp rx=0 tx=0).stop 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f26a3678640 1 -- 172.21.15.47:0/53836454 shutdown_connections 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f26a3678640 1 --2- 172.21.15.47:0/53836454 >> v2:172.21.15.47:3300/0 conn(0x7f269c094760 0x7f269c134760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f26a3678640 1 --2- 172.21.15.47:0/53836454 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c096b00 0x7f269c096ee0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f26a3678640 1 -- 172.21.15.47:0/53836454 >> 172.21.15.47:0/53836454 conn(0x7f269c09ddd0 msgr2=0x7f269c09e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f26a3678640 1 -- 172.21.15.47:0/53836454 shutdown_connections 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.138+0000 7f26a3678640 1 -- 172.21.15.47:0/53836454 wait complete. 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.139+0000 7f26a3678640 1 Processor -- start 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.139+0000 7f26a3678640 1 -- start start 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.139+0000 7f26a3678640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f269c094760 0x7f269c1c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.139+0000 7f26a3678640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f269c096b00 0x7f269c1c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.139+0000 7f26a3678640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c097420 0x7f269c1c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.139+0000 7f26a3678640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f269c097880 con 0x7f269c094760 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.139+0000 7f26a3678640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f269c095340 con 0x7f269c096b00 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.139+0000 7f26a3678640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f269c094b90 con 0x7f269c097420 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a1414640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f269c094760 0x7f269c1c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a1c15640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c097420 0x7f269c1c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a0c13640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f269c096b00 0x7f269c1c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a1c15640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c097420 0x7f269c1c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36984/0 (socket says 172.21.15.47:36984) 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a1c15640 1 -- 172.21.15.47:0/794698203 learned_addr learned my addr 172.21.15.47:0/794698203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a1414640 1 --2- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f269c094760 0x7f269c1c45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57970/0 (socket says 172.21.15.47:57970) 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a0c13640 1 -- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c097420 msgr2=0x7f269c1c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a0c13640 1 --2- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c097420 0x7f269c1c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a0c13640 1 -- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f269c094760 msgr2=0x7f269c1c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a0c13640 1 --2- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f269c094760 0x7f269c1c45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a0c13640 1 -- 172.21.15.47:0/794698203 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c129070 con 0x7f269c096b00 2024-09-15T21:57:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a1c15640 1 --2- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c097420 0x7f269c1c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a1414640 1 --2- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f269c094760 0x7f269c1c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a0c13640 1 --2- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f269c096b00 0x7f269c1c4b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f268c00c430 tx=0x7f268c00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f2691ffb640 1 -- 172.21.15.47:0/794698203 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c0074e0 con 0x7f269c096b00 2024-09-15T21:57:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f2691ffb640 1 -- 172.21.15.47:0/794698203 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268c007680 con 0x7f269c096b00 2024-09-15T21:57:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f2691ffb640 1 -- 172.21.15.47:0/794698203 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c0049e0 con 0x7f269c096b00 2024-09-15T21:57:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.140+0000 7f26a3678640 1 -- 172.21.15.47:0/794698203 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c08f820 con 0x7f269c096b00 2024-09-15T21:57:02.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.141+0000 7f26a3678640 1 -- 172.21.15.47:0/794698203 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c08ec70 con 0x7f269c096b00 2024-09-15T21:57:02.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.142+0000 7f2691ffb640 1 -- 172.21.15.47:0/794698203 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f268c01a020 con 0x7f269c096b00 2024-09-15T21:57:02.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.142+0000 7f26a3678640 1 -- 172.21.15.47:0/794698203 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269c096ee0 con 0x7f269c096b00 2024-09-15T21:57:02.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.142+0000 7f2691ffb640 1 -- 172.21.15.47:0/794698203 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f268c05d130 con 0x7f269c096b00 2024-09-15T21:57:02.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.146+0000 7f2691ffb640 1 -- 172.21.15.47:0/794698203 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f268c03c280 con 0x7f269c096b00 2024-09-15T21:57:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.293+0000 7f26a3678640 1 -- 172.21.15.47:0/794698203 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f269c075d40 con 0x7f2680047fd0 2024-09-15T21:57:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.294+0000 7f2691ffb640 1 -- 172.21.15.47:0/794698203 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (unknown 2057319476 0 0) 0x7f269c075d40 con 0x7f2680047fd0 2024-09-15T21:57:02.296 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T21:57:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.295+0000 7f26a3678640 1 -- 172.21.15.47:0/794698203 >> v1:172.21.15.47:6800/1790418198 conn(0x7f2680047fd0 legacy=0x7f268004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.295+0000 7f26a3678640 1 -- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f269c096b00 msgr2=0x7f269c1c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.295+0000 7f26a3678640 1 --2- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f269c096b00 0x7f269c1c4b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f268c00c430 tx=0x7f268c00c8f0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.295+0000 7f26a3678640 1 -- 172.21.15.47:0/794698203 shutdown_connections 2024-09-15T21:57:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.295+0000 7f26a3678640 1 --2- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f269c097420 0x7f269c1c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.295+0000 7f26a3678640 1 --2- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f269c096b00 0x7f269c1c4b00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.295+0000 7f26a3678640 1 --2- 172.21.15.47:0/794698203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f269c094760 0x7f269c1c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.295+0000 7f26a3678640 1 -- 172.21.15.47:0/794698203 >> 172.21.15.47:0/794698203 conn(0x7f269c09ddd0 msgr2=0x7f269c099480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.295+0000 7f26a3678640 1 -- 172.21.15.47:0/794698203 shutdown_connections 2024-09-15T21:57:02.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.295+0000 7f26a3678640 1 -- 172.21.15.47:0/794698203 wait complete. 2024-09-15T21:57:02.308 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T21:57:02.309 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:57:02.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b2ec08640 1 Processor -- start 2024-09-15T21:57:02.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b2ec08640 1 -- start start 2024-09-15T21:57:02.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b2ec08640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b2ec08640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6b2806d560 0x7f6b28097830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b2ec08640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b2806f2e0 con 0x7f6b28130cc0 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b2ec08640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6b2806f110 con 0x7f6b2806d560 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b2ec08640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f6b28070b70 con 0x7f6b2806d180 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b2c9a4640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b2c9a4640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45398/0 (socket says 172.21.15.47:45398) 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b2c9a4640 1 -- 172.21.15.47:0/2075066868 learned_addr learned my addr 172.21.15.47:0/2075066868 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.412+0000 7f6b27fff640 1 --2- 172.21.15.47:0/2075066868 >> v2:172.21.15.47:3300/0 conn(0x7f6b2806d560 0x7f6b28097830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.413+0000 7f6b2c9a4640 1 -- 172.21.15.47:0/2075066868 >> v2:172.21.15.47:3300/0 conn(0x7f6b2806d560 msgr2=0x7f6b28097830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.413+0000 7f6b2c9a4640 1 --2- 172.21.15.47:0/2075066868 >> v2:172.21.15.47:3300/0 conn(0x7f6b2806d560 0x7f6b28097830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.413+0000 7f6b2c9a4640 1 -- 172.21.15.47:0/2075066868 >> v1:172.21.15.47:6789/0 conn(0x7f6b28130cc0 legacy=0x7f6b281330b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.413+0000 7f6b2c9a4640 1 -- 172.21.15.47:0/2075066868 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2806fa40 con 0x7f6b2806d180 2024-09-15T21:57:02.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.413+0000 7f6b27fff640 1 --2- 172.21.15.47:0/2075066868 >> v2:172.21.15.47:3300/0 conn(0x7f6b2806d560 0x7f6b28097830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.413+0000 7f6b2c9a4640 1 --2- 172.21.15.47:0/2075066868 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280972f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00bb10 tx=0x7f6b1c02c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=252655085a9855b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.413+0000 7f6b277fe640 1 -- 172.21.15.47:0/2075066868 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c032700 con 0x7f6b2806d180 2024-09-15T21:57:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.414+0000 7f6b277fe640 1 -- 172.21.15.47:0/2075066868 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c0328a0 con 0x7f6b2806d180 2024-09-15T21:57:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.414+0000 7f6b277fe640 1 -- 172.21.15.47:0/2075066868 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c032c70 con 0x7f6b2806d180 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.414+0000 7f6b2ec08640 1 -- 172.21.15.47:0/2075066868 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 msgr2=0x7f6b280972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.414+0000 7f6b2ec08640 1 --2- 172.21.15.47:0/2075066868 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280972f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00bb10 tx=0x7f6b1c02c800 comp rx=0 tx=0).stop 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.414+0000 7f6b2ec08640 1 -- 172.21.15.47:0/2075066868 shutdown_connections 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.414+0000 7f6b2ec08640 1 --2- 172.21.15.47:0/2075066868 >> v2:172.21.15.47:3300/0 conn(0x7f6b2806d560 0x7f6b28097830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.414+0000 7f6b2ec08640 1 --2- 172.21.15.47:0/2075066868 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280972f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.414+0000 7f6b2ec08640 1 -- 172.21.15.47:0/2075066868 >> 172.21.15.47:0/2075066868 conn(0x7f6b2809db10 msgr2=0x7f6b2809df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.414+0000 7f6b2ec08640 1 -- 172.21.15.47:0/2075066868 shutdown_connections 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.414+0000 7f6b2ec08640 1 -- 172.21.15.47:0/2075066868 wait complete. 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.415+0000 7f6b2ec08640 1 Processor -- start 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.415+0000 7f6b2ec08640 1 -- start start 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.415+0000 7f6b2ec08640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.415+0000 7f6b2ec08640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b2806d560 0x7f6b28095900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2ec08640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b28130cc0 0x7f6b28095e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2ec08640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2806db90 con 0x7f6b2806d560 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2ec08640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2806da10 con 0x7f6b2806d180 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2ec08640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f6b2806dd10 con 0x7f6b28130cc0 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2c9a4640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b27fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b2806d560 0x7f6b28095900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2c9a4640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45408/0 (socket says 172.21.15.47:45408) 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b27fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b2806d560 0x7f6b28095900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57982/0 (socket says 172.21.15.47:57982) 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2c9a4640 1 -- 172.21.15.47:0/1702748318 learned_addr learned my addr 172.21.15.47:0/1702748318 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2d1a5640 1 --2- 172.21.15.47:0/1702748318 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b28130cc0 0x7f6b28095e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2c9a4640 1 -- 172.21.15.47:0/1702748318 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b28130cc0 msgr2=0x7f6b28095e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2c9a4640 1 --2- 172.21.15.47:0/1702748318 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f6b28130cc0 0x7f6b28095e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2c9a4640 1 -- 172.21.15.47:0/1702748318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b2806d560 msgr2=0x7f6b28095900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2c9a4640 1 --2- 172.21.15.47:0/1702748318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b2806d560 0x7f6b28095900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2c9a4640 1 -- 172.21.15.47:0/1702748318 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b28127d30 con 0x7f6b2806d180 2024-09-15T21:57:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b27fff640 1 --2- 172.21.15.47:0/1702748318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6b2806d560 0x7f6b28095900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2c9a4640 1 --2- 172.21.15.47:0/1702748318 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280953c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c02c840 tx=0x7f6b1c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b257fa640 1 -- 172.21.15.47:0/1702748318 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c041070 con 0x7f6b2806d180 2024-09-15T21:57:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b257fa640 1 -- 172.21.15.47:0/1702748318 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c034b50 con 0x7f6b2806d180 2024-09-15T21:57:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b28127330 con 0x7f6b2806d180 2024-09-15T21:57:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b257fa640 1 -- 172.21.15.47:0/1702748318 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c03d440 con 0x7f6b2806d180 2024-09-15T21:57:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.416+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b28130890 con 0x7f6b2806d180 2024-09-15T21:57:02.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.417+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b28128510 con 0x7f6b2806d180 2024-09-15T21:57:02.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.417+0000 7f6b257fa640 1 -- 172.21.15.47:0/1702748318 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f6b1c044070 con 0x7f6b2806d180 2024-09-15T21:57:02.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.418+0000 7f6b257fa640 1 -- 172.21.15.47:0/1702748318 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f6b1c07e3b0 con 0x7f6b2806d180 2024-09-15T21:57:02.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.418+0000 7f6b257fa640 1 -- 172.21.15.47:0/1702748318 --> v1:172.21.15.47:6805/4287109391 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b1c041e70 con 0x7f6b0c04e850 2024-09-15T21:57:02.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.418+0000 7f6b257fa640 1 -- 172.21.15.47:0/1702748318 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6b1c085050 con 0x7f6b2806d180 2024-09-15T21:57:02.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.419+0000 7f6b257fa640 1 -- 172.21.15.47:0/1702748318 <== osd.1 v1:172.21.15.47:6805/4287109391 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f6b1c041e70 con 0x7f6b0c04e850 2024-09-15T21:57:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.431+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 --> v1:172.21.15.47:6805/4287109391 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6b28071790 con 0x7f6b0c04e850 2024-09-15T21:57:02.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.432+0000 7f6b257fa640 1 -- 172.21.15.47:0/1702748318 <== osd.1 v1:172.21.15.47:6805/4287109391 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (unknown 0 0 125856537) 0x7f6b28071790 con 0x7f6b0c04e850 2024-09-15T21:57:02.434 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T21:57:02.434 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-15T21:57:02.434 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T21:57:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.432+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 >> v1:172.21.15.47:6805/4287109391 conn(0x7f6b0c04e850 legacy=0x7f6b0c050c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.432+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 >> v1:172.21.15.47:6800/1790418198 conn(0x7f6b0c047fd0 legacy=0x7f6b0c04a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.432+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 msgr2=0x7f6b280953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.432+0000 7f6b2ec08640 1 --2- 172.21.15.47:0/1702748318 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f6b2806d180 0x7f6b280953c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c02c840 tx=0x7f6b1c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.432+0000 7f6b2d1a5640 1 -- 172.21.15.47:0/1702748318 reap_dead start 2024-09-15T21:57:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.432+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 shutdown_connections 2024-09-15T21:57:02.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.432+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 >> 172.21.15.47:0/1702748318 conn(0x7f6b2809db10 msgr2=0x7f6b28070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.432+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 shutdown_connections 2024-09-15T21:57:02.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.432+0000 7f6b2ec08640 1 -- 172.21.15.47:0/1702748318 wait complete. 2024-09-15T21:57:02.444 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T21:57:02.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.544+0000 7fc4ecf33640 1 Processor -- start 2024-09-15T21:57:02.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.544+0000 7fc4ecf33640 1 -- start start 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.544+0000 7fc4ecf33640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8097560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.544+0000 7fc4ecf33640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097aa0 0x7fc4e8131d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.544+0000 7fc4ecf33640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e806cfa0 con 0x7fc4e8097ea0 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.544+0000 7fc4ecf33640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e806d6b0 con 0x7fc4e8097aa0 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.544+0000 7fc4ecf33640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fc4e80990c0 con 0x7fc4e8097180 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.545+0000 7fc4e659c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8097560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.545+0000 7fc4e5d9b640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097aa0 0x7fc4e8131d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.545+0000 7fc4e659c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8097560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:36998/0 (socket says 172.21.15.47:36998) 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.545+0000 7fc4e659c640 1 -- 172.21.15.47:0/767689404 learned_addr learned my addr 172.21.15.47:0/767689404 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.545+0000 7fc4e5d9b640 1 -- 172.21.15.47:0/767689404 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 msgr2=0x7fc4e8097560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:02.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.545+0000 7fc4e5d9b640 1 --2- 172.21.15.47:0/767689404 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8097560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.545+0000 7fc4e5d9b640 1 -- 172.21.15.47:0/767689404 >> v1:172.21.15.47:6789/0 conn(0x7fc4e8097ea0 legacy=0x7fc4e81342d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.545+0000 7fc4e559a640 1 -- 172.21.15.47:0/767689404 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2320080833 0 0) 0x7fc4e806cfa0 con 0x7fc4e8097ea0 2024-09-15T21:57:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.545+0000 7fc4e5d9b640 1 -- 172.21.15.47:0/767689404 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e80989d0 con 0x7fc4e8097aa0 2024-09-15T21:57:02.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.545+0000 7fc4e659c640 1 --2- 172.21.15.47:0/767689404 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8097560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4e5d9b640 1 --2- 172.21.15.47:0/767689404 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097aa0 0x7fc4e8131d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0097d0 tx=0x7fc4dc02a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6345b56eddffab83 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4e559a640 1 -- 172.21.15.47:0/767689404 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc037070 con 0x7fc4e8097aa0 2024-09-15T21:57:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4e559a640 1 -- 172.21.15.47:0/767689404 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4dc02ad90 con 0x7fc4e8097aa0 2024-09-15T21:57:02.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4e559a640 1 -- 172.21.15.47:0/767689404 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc032660 con 0x7fc4e8097aa0 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4ecf33640 1 -- 172.21.15.47:0/767689404 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097aa0 msgr2=0x7fc4e8131d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4ecf33640 1 --2- 172.21.15.47:0/767689404 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097aa0 0x7fc4e8131d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0097d0 tx=0x7fc4dc02a040 comp rx=0 tx=0).stop 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4ecf33640 1 -- 172.21.15.47:0/767689404 shutdown_connections 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4ecf33640 1 --2- 172.21.15.47:0/767689404 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097aa0 0x7fc4e8131d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4ecf33640 1 --2- 172.21.15.47:0/767689404 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8097560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4ecf33640 1 -- 172.21.15.47:0/767689404 >> 172.21.15.47:0/767689404 conn(0x7fc4e809b700 msgr2=0x7fc4e809bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4ecf33640 1 -- 172.21.15.47:0/767689404 shutdown_connections 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.546+0000 7fc4ecf33640 1 -- 172.21.15.47:0/767689404 wait complete. 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.547+0000 7fc4ecf33640 1 Processor -- start 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.547+0000 7fc4ecf33640 1 -- start start 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4ecf33640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8123cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4ecf33640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e8097aa0 0x7fc4e8124200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4ecf33640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097ea0 0x7fc4e8124740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4ecf33640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e8135b70 con 0x7fc4e8097aa0 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4ecf33640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e81359f0 con 0x7fc4e8097ea0 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4ecf33640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fc4e8135cf0 con 0x7fc4e8097180 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e659c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8123cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e5d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e8097aa0 0x7fc4e8124200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e6d9d640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097ea0 0x7fc4e8124740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e659c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8123cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37006/0 (socket says 172.21.15.47:37006) 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e659c640 1 -- 172.21.15.47:0/3601558616 learned_addr learned my addr 172.21.15.47:0/3601558616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e659c640 1 -- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097ea0 msgr2=0x7fc4e8124740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e659c640 1 --2- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097ea0 0x7fc4e8124740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e659c640 1 -- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e8097aa0 msgr2=0x7fc4e8124200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e659c640 1 --2- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e8097aa0 0x7fc4e8124200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e659c640 1 -- 172.21.15.47:0/3601558616 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e8073080 con 0x7fc4e8097180 2024-09-15T21:57:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e6d9d640 1 --2- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097ea0 0x7fc4e8124740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.548+0000 7fc4e659c640 1 --2- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8123cc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800e9d0 tx=0x7fc4d800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.549+0000 7fc4d6ffd640 1 -- 172.21.15.47:0/3601558616 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8017070 con 0x7fc4e8097180 2024-09-15T21:57:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.549+0000 7fc4d6ffd640 1 -- 172.21.15.47:0/3601558616 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d800ca60 con 0x7fc4e8097180 2024-09-15T21:57:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.549+0000 7fc4ecf33640 1 -- 172.21.15.47:0/3601558616 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e8095c30 con 0x7fc4e8097180 2024-09-15T21:57:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.549+0000 7fc4d6ffd640 1 -- 172.21.15.47:0/3601558616 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800ce10 con 0x7fc4e8097180 2024-09-15T21:57:02.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.549+0000 7fc4ecf33640 1 -- 172.21.15.47:0/3601558616 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e80952b0 con 0x7fc4e8097180 2024-09-15T21:57:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.550+0000 7fc4d6ffd640 1 -- 172.21.15.47:0/3601558616 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fc4d801a070 con 0x7fc4e8097180 2024-09-15T21:57:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.550+0000 7fc4ecf33640 1 -- 172.21.15.47:0/3601558616 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e806c160 con 0x7fc4e8097180 2024-09-15T21:57:02.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.550+0000 7fc4d6ffd640 1 -- 172.21.15.47:0/3601558616 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fc4d805d900 con 0x7fc4e8097180 2024-09-15T21:57:02.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.554+0000 7fc4d6ffd640 1 -- 172.21.15.47:0/3601558616 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7fc4e806c160 con 0x7fc4e8097180 2024-09-15T21:57:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.702+0000 7fc4ecf33640 1 -- 172.21.15.47:0/3601558616 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fc4e8068fa0 con 0x7fc4c8047fd0 2024-09-15T21:57:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.702+0000 7fc4d6ffd640 1 -- 172.21.15.47:0/3601558616 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (unknown 489605429 0 0) 0x7fc4e8068fa0 con 0x7fc4c8047fd0 2024-09-15T21:57:02.705 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T21:57:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.704+0000 7fc4ecf33640 1 -- 172.21.15.47:0/3601558616 >> v1:172.21.15.47:6800/1790418198 conn(0x7fc4c8047fd0 legacy=0x7fc4c804a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.704+0000 7fc4ecf33640 1 -- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 msgr2=0x7fc4e8123cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.704+0000 7fc4ecf33640 1 --2- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8123cc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800e9d0 tx=0x7fc4d800ee90 comp rx=0 tx=0).stop 2024-09-15T21:57:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.704+0000 7fc4ecf33640 1 -- 172.21.15.47:0/3601558616 shutdown_connections 2024-09-15T21:57:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.704+0000 7fc4ecf33640 1 --2- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fc4e8097ea0 0x7fc4e8124740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.704+0000 7fc4ecf33640 1 --2- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4e8097aa0 0x7fc4e8124200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.704+0000 7fc4ecf33640 1 --2- 172.21.15.47:0/3601558616 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fc4e8097180 0x7fc4e8123cc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.704+0000 7fc4ecf33640 1 -- 172.21.15.47:0/3601558616 >> 172.21.15.47:0/3601558616 conn(0x7fc4e809b700 msgr2=0x7fc4e8071900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.704+0000 7fc4ecf33640 1 -- 172.21.15.47:0/3601558616 shutdown_connections 2024-09-15T21:57:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.704+0000 7fc4ecf33640 1 -- 172.21.15.47:0/3601558616 wait complete. 2024-09-15T21:57:02.717 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T21:57:02.717 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:57:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.820+0000 7fdb7e70f640 1 Processor -- start 2024-09-15T21:57:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.820+0000 7fdb7e70f640 1 -- start start 2024-09-15T21:57:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb7e70f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb7809b9b0 0x7fdb7806de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb7e70f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 0x7fdb78137110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb7e70f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb7809c350 con 0x7fdb7806f9f0 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb7e70f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7806ea90 con 0x7fdb7809b9b0 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb7e70f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fdb7806ec10 con 0x7fdb78134d20 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb7ccac640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 0x7fdb78137110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb77fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb7809b9b0 0x7fdb7806de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb7ccac640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 0x7fdb78137110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37022/0 (socket says 172.21.15.47:37022) 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb77fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdb7809b9b0 0x7fdb7806de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58002/0 (socket says 172.21.15.47:58002) 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb7ccac640 1 -- 172.21.15.47:0/393947763 learned_addr learned my addr 172.21.15.47:0/393947763 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb77fff640 1 -- 172.21.15.47:0/393947763 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 msgr2=0x7fdb78137110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb77fff640 1 --2- 172.21.15.47:0/393947763 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 0x7fdb78137110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb77fff640 1 -- 172.21.15.47:0/393947763 >> v1:172.21.15.47:6789/0 conn(0x7fdb7806f9f0 legacy=0x7fdb7806e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb77fff640 1 -- 172.21.15.47:0/393947763 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb78138660 con 0x7fdb7809b9b0 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb76ffd640 1 -- 172.21.15.47:0/393947763 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4230820339 0 0) 0x7fdb7809c350 con 0x7fdb7806f9f0 2024-09-15T21:57:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.821+0000 7fdb7ccac640 1 --2- 172.21.15.47:0/393947763 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 0x7fdb78137110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.822+0000 7fdb77fff640 1 --2- 172.21.15.47:0/393947763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb7809b9b0 0x7fdb7806de40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdb680097a0 tx=0x7fdb6802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739827c24655c48a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.822+0000 7fdb76ffd640 1 -- 172.21.15.47:0/393947763 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68037070 con 0x7fdb7809b9b0 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.822+0000 7fdb76ffd640 1 -- 172.21.15.47:0/393947763 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6802adb0 con 0x7fdb7809b9b0 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.822+0000 7fdb76ffd640 1 -- 172.21.15.47:0/393947763 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68032660 con 0x7fdb7809b9b0 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.823+0000 7fdb7e70f640 1 -- 172.21.15.47:0/393947763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb7809b9b0 msgr2=0x7fdb7806de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.823+0000 7fdb7e70f640 1 --2- 172.21.15.47:0/393947763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb7809b9b0 0x7fdb7806de40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdb680097a0 tx=0x7fdb6802a060 comp rx=0 tx=0).stop 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.823+0000 7fdb7e70f640 1 -- 172.21.15.47:0/393947763 shutdown_connections 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.823+0000 7fdb7e70f640 1 --2- 172.21.15.47:0/393947763 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 0x7fdb78137110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.823+0000 7fdb7e70f640 1 --2- 172.21.15.47:0/393947763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb7809b9b0 0x7fdb7806de40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.823+0000 7fdb7e70f640 1 -- 172.21.15.47:0/393947763 >> 172.21.15.47:0/393947763 conn(0x7fdb7809caa0 msgr2=0x7fdb7809ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.823+0000 7fdb7e70f640 1 -- 172.21.15.47:0/393947763 shutdown_connections 2024-09-15T21:57:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.823+0000 7fdb7e70f640 1 -- 172.21.15.47:0/393947763 wait complete. 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.824+0000 7fdb7e70f640 1 Processor -- start 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.824+0000 7fdb7e70f640 1 -- start start 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.824+0000 7fdb7e70f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb7806f9f0 0x7fdb78133800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.824+0000 7fdb7e70f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb7809b9b0 0x7fdb78133d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.824+0000 7fdb7e70f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 0x7fdb78096180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.824+0000 7fdb7e70f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781379e0 con 0x7fdb7809b9b0 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.824+0000 7fdb7e70f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7fdb78137860 con 0x7fdb7806f9f0 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.824+0000 7fdb7e70f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7fdb78137b60 con 0x7fdb78134d20 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb7806f9f0 0x7fdb78133800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb7809b9b0 0x7fdb78133d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb7ccac640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 0x7fdb78096180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb7809b9b0 0x7fdb78133d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58006/0 (socket says 172.21.15.47:58006) 2024-09-15T21:57:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb7ccac640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 0x7fdb78096180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:37036/0 (socket says 172.21.15.47:37036) 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb777fe640 1 -- 172.21.15.47:0/3364588047 learned_addr learned my addr 172.21.15.47:0/3364588047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb7806f9f0 0x7fdb78133800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45430/0 (socket says 172.21.15.47:45430) 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb777fe640 1 -- 172.21.15.47:0/3364588047 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 msgr2=0x7fdb78096180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb777fe640 1 --2- 172.21.15.47:0/3364588047 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7fdb78134d20 0x7fdb78096180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb777fe640 1 -- 172.21.15.47:0/3364588047 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb7806f9f0 msgr2=0x7fdb78133800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb777fe640 1 --2- 172.21.15.47:0/3364588047 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb7806f9f0 0x7fdb78133800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb777fe640 1 -- 172.21.15.47:0/3364588047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb781250f0 con 0x7fdb7809b9b0 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb77fff640 1 --2- 172.21.15.47:0/3364588047 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7fdb7806f9f0 0x7fdb78133800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb777fe640 1 --2- 172.21.15.47:0/3364588047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb7809b9b0 0x7fdb78133d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00c460 tx=0x7fdb6c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb74ff9640 1 -- 172.21.15.47:0/3364588047 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c005480 con 0x7fdb7809b9b0 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb74ff9640 1 -- 172.21.15.47:0/3364588047 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6c0035f0 con 0x7fdb7809b9b0 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb78134280 con 0x7fdb7809b9b0 2024-09-15T21:57:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb74ff9640 1 -- 172.21.15.47:0/3364588047 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0039e0 con 0x7fdb7809b9b0 2024-09-15T21:57:02.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.825+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb780985b0 con 0x7fdb7809b9b0 2024-09-15T21:57:02.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.826+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb78124d20 con 0x7fdb7809b9b0 2024-09-15T21:57:02.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.826+0000 7fdb74ff9640 1 -- 172.21.15.47:0/3364588047 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7fdb6c003b80 con 0x7fdb7809b9b0 2024-09-15T21:57:02.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.827+0000 7fdb74ff9640 1 -- 172.21.15.47:0/3364588047 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7fdb6c05d4e0 con 0x7fdb7809b9b0 2024-09-15T21:57:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.827+0000 7fdb74ff9640 1 -- 172.21.15.47:0/3364588047 --> v1:172.21.15.47:6809/2903715449 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb6c005480 con 0x7fdb6404ea60 2024-09-15T21:57:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.827+0000 7fdb74ff9640 1 -- 172.21.15.47:0/3364588047 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdb6c064050 con 0x7fdb7809b9b0 2024-09-15T21:57:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.828+0000 7fdb74ff9640 1 -- 172.21.15.47:0/3364588047 <== osd.2 v1:172.21.15.47:6809/2903715449 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fdb6c005480 con 0x7fdb6404ea60 2024-09-15T21:57:02.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.840+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 --> v1:172.21.15.47:6809/2903715449 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdb780730f0 con 0x7fdb6404ea60 2024-09-15T21:57:02.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.840+0000 7fdb74ff9640 1 -- 172.21.15.47:0/3364588047 <== osd.2 v1:172.21.15.47:6809/2903715449 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (unknown 0 0 125856537) 0x7fdb780730f0 con 0x7fdb6404ea60 2024-09-15T21:57:02.843 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T21:57:02.843 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-15T21:57:02.843 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T21:57:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.841+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 >> v1:172.21.15.47:6809/2903715449 conn(0x7fdb6404ea60 legacy=0x7fdb64050ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.841+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 >> v1:172.21.15.47:6800/1790418198 conn(0x7fdb64048170 legacy=0x7fdb6404a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.841+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb7809b9b0 msgr2=0x7fdb78133d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.841+0000 7fdb7e70f640 1 --2- 172.21.15.47:0/3364588047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdb7809b9b0 0x7fdb78133d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00c460 tx=0x7fdb6c00c920 comp rx=0 tx=0).stop 2024-09-15T21:57:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.841+0000 7fdb7ccac640 1 -- 172.21.15.47:0/3364588047 reap_dead start 2024-09-15T21:57:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.841+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 shutdown_connections 2024-09-15T21:57:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.841+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 >> 172.21.15.47:0/3364588047 conn(0x7fdb7809caa0 msgr2=0x7fdb78072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.841+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 shutdown_connections 2024-09-15T21:57:02.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.841+0000 7fdb7e70f640 1 -- 172.21.15.47:0/3364588047 wait complete. 2024-09-15T21:57:02.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T21:57:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5af6303640 1 Processor -- start 2024-09-15T21:57:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5af6303640 1 -- start start 2024-09-15T21:57:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5af6303640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af00975f0 0x7f5af0131740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5af6303640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5af0131c80 0x7f5af0134070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5af6303640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af006cfd0 con 0x7f5af0096b00 2024-09-15T21:57:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5af6303640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5af006e1a0 con 0x7f5af0131c80 2024-09-15T21:57:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5af6303640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f5af006ccf0 con 0x7f5af00975f0 2024-09-15T21:57:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af00975f0 0x7f5af0131740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5af48a0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5af0131c80 0x7f5af0134070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af00975f0 0x7f5af0131740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45442/0 (socket says 172.21.15.47:45442) 2024-09-15T21:57:02.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5aeffff640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5af0096b00 0x7f5af0096ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34286/0 (socket says 172.21.15.47:34286) 2024-09-15T21:57:02.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.953+0000 7f5ae7fff640 1 -- 172.21.15.47:0/1534503155 learned_addr learned my addr 172.21.15.47:0/1534503155 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.954+0000 7f5aef7fe640 1 -- 172.21.15.47:0/1534503155 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3596300292 0 0) 0x7f5af006cfd0 con 0x7f5af0096b00 2024-09-15T21:57:02.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.954+0000 7f5aef7fe640 1 -- 172.21.15.47:0/1534503155 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5adc003520 con 0x7f5af0096b00 2024-09-15T21:57:02.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.954+0000 7f5af48a0640 1 -- 172.21.15.47:0/1534503155 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af00975f0 msgr2=0x7f5af0131740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:02.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.954+0000 7f5af48a0640 1 --2- 172.21.15.47:0/1534503155 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af00975f0 0x7f5af0131740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.954+0000 7f5af48a0640 1 -- 172.21.15.47:0/1534503155 >> v1:172.21.15.47:6789/0 conn(0x7f5af0096b00 legacy=0x7f5af0096ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.954+0000 7f5af48a0640 1 -- 172.21.15.47:0/1534503155 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af006de90 con 0x7f5af0131c80 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.954+0000 7f5ae7fff640 1 --2- 172.21.15.47:0/1534503155 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af00975f0 0x7f5af0131740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.954+0000 7f5af48a0640 1 --2- 172.21.15.47:0/1534503155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af0131c80 0x7f5af0134070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009d90 tx=0x7f5ae802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee887ecccc6e90c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5aef7fe640 1 -- 172.21.15.47:0/1534503155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8007740 con 0x7f5af0131c80 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5aef7fe640 1 -- 172.21.15.47:0/1534503155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae80078e0 con 0x7f5af0131c80 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5aef7fe640 1 -- 172.21.15.47:0/1534503155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8007cb0 con 0x7f5af0131c80 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5af6303640 1 -- 172.21.15.47:0/1534503155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af0131c80 msgr2=0x7f5af0134070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5af6303640 1 --2- 172.21.15.47:0/1534503155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af0131c80 0x7f5af0134070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009d90 tx=0x7f5ae802a530 comp rx=0 tx=0).stop 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5af6303640 1 -- 172.21.15.47:0/1534503155 shutdown_connections 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5af6303640 1 --2- 172.21.15.47:0/1534503155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af0131c80 0x7f5af0134070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5af6303640 1 --2- 172.21.15.47:0/1534503155 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af00975f0 0x7f5af0131740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5af6303640 1 -- 172.21.15.47:0/1534503155 >> 172.21.15.47:0/1534503155 conn(0x7f5af009ddd0 msgr2=0x7f5af009e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:02.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5af6303640 1 -- 172.21.15.47:0/1534503155 shutdown_connections 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.955+0000 7f5af6303640 1 -- 172.21.15.47:0/1534503155 wait complete. 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.956+0000 7f5af6303640 1 Processor -- start 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.956+0000 7f5af6303640 1 -- start start 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.956+0000 7f5af6303640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af0096b00 0x7f5af0070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.956+0000 7f5af6303640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af00975f0 0x7f5af00713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.956+0000 7f5af6303640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5af0131c80 0x7f5af0071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.956+0000 7f5af6303640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01345b0 con 0x7f5af00975f0 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.956+0000 7f5af6303640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f5af0095340 con 0x7f5af0096b00 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.956+0000 7f5af6303640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f5af0134730 con 0x7f5af0131c80 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af0096b00 0x7f5af0070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af00975f0 0x7f5af00713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5af48a0640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5af0131c80 0x7f5af0071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af00975f0 0x7f5af00713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58018/0 (socket says 172.21.15.47:58018) 2024-09-15T21:57:02.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5ae7fff640 1 -- 172.21.15.47:0/2715698175 learned_addr learned my addr 172.21.15.47:0/2715698175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5ae7fff640 1 -- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5af0131c80 msgr2=0x7f5af0071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5ae7fff640 1 --2- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5af0131c80 0x7f5af0071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5ae7fff640 1 -- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af0096b00 msgr2=0x7f5af0070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5ae7fff640 1 --2- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af0096b00 0x7f5af0070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5ae7fff640 1 -- 172.21.15.47:0/2715698175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af0123200 con 0x7f5af00975f0 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5aeffff640 1 --2- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af0096b00 0x7f5af0070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5ae7fff640 1 --2- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af00975f0 0x7f5af00713f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800ea00 tx=0x7f5ad800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5aed7fa640 1 -- 172.21.15.47:0/2715698175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8017070 con 0x7f5af00975f0 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5aed7fa640 1 -- 172.21.15.47:0/2715698175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad800ca60 con 0x7f5af00975f0 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5af6303640 1 -- 172.21.15.47:0/2715698175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af0072920 con 0x7f5af00975f0 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5aed7fa640 1 -- 172.21.15.47:0/2715698175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800ce10 con 0x7f5af00975f0 2024-09-15T21:57:02.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.957+0000 7f5af6303640 1 -- 172.21.15.47:0/2715698175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af0071e70 con 0x7f5af00975f0 2024-09-15T21:57:02.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.958+0000 7f5aed7fa640 1 -- 172.21.15.47:0/2715698175 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f5ad801a070 con 0x7f5af00975f0 2024-09-15T21:57:02.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.958+0000 7f5af6303640 1 -- 172.21.15.47:0/2715698175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af0096ee0 con 0x7f5af00975f0 2024-09-15T21:57:02.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.959+0000 7f5aed7fa640 1 -- 172.21.15.47:0/2715698175 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f5ad805d900 con 0x7f5af00975f0 2024-09-15T21:57:02.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:02.962+0000 7f5aed7fa640 1 -- 172.21.15.47:0/2715698175 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f5ad805c9b0 con 0x7f5af00975f0 2024-09-15T21:57:03.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.109+0000 7f5af6303640 1 -- 172.21.15.47:0/2715698175 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f5af0123d80 con 0x7f5ad0047fd0 2024-09-15T21:57:03.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.109+0000 7f5aed7fa640 1 -- 172.21.15.47:0/2715698175 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (unknown 3049115190 0 0) 0x7f5ae000e9d0 con 0x7f5ad0047fd0 2024-09-15T21:57:03.112 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T21:57:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.111+0000 7f5af6303640 1 -- 172.21.15.47:0/2715698175 >> v1:172.21.15.47:6800/1790418198 conn(0x7f5ad0047fd0 legacy=0x7f5ad004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.111+0000 7f5af6303640 1 -- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af00975f0 msgr2=0x7f5af00713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.111+0000 7f5af6303640 1 --2- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af00975f0 0x7f5af00713f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800ea00 tx=0x7f5ad800eec0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.111+0000 7f5af6303640 1 -- 172.21.15.47:0/2715698175 shutdown_connections 2024-09-15T21:57:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.111+0000 7f5af6303640 1 --2- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f5af0131c80 0x7f5af0071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.111+0000 7f5af6303640 1 --2- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af00975f0 0x7f5af00713f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.111+0000 7f5af6303640 1 --2- 172.21.15.47:0/2715698175 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f5af0096b00 0x7f5af0070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.111+0000 7f5af6303640 1 -- 172.21.15.47:0/2715698175 >> 172.21.15.47:0/2715698175 conn(0x7f5af009ddd0 msgr2=0x7f5af012cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.111+0000 7f5af6303640 1 -- 172.21.15.47:0/2715698175 shutdown_connections 2024-09-15T21:57:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.111+0000 7f5af6303640 1 -- 172.21.15.47:0/2715698175 wait complete. 2024-09-15T21:57:03.124 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T21:57:03.125 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:57:03.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.227+0000 7f44b46cb640 1 Processor -- start 2024-09-15T21:57:03.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.227+0000 7f44b46cb640 1 -- start start 2024-09-15T21:57:03.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b46cb640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44ac06e560 0x7f44ac12d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b46cb640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12da00 0x7f44ac12e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b46cb640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44ac099460 con 0x7f44ac12de60 2024-09-15T21:57:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b46cb640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f44ac06f810 con 0x7f44ac06e560 2024-09-15T21:57:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b46cb640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac0998a0 con 0x7f44ac12da00 2024-09-15T21:57:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b1c66640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12da00 0x7f44ac12e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b2467640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f44ac06e560 0x7f44ac12d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b1c66640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12da00 0x7f44ac12e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45466/0 (socket says 172.21.15.47:45466) 2024-09-15T21:57:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b1c66640 1 -- 172.21.15.47:0/3244446287 learned_addr learned my addr 172.21.15.47:0/3244446287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b2c68640 1 --1- 172.21.15.47:0/3244446287 >> v1:172.21.15.47:6789/0 conn(0x7f44ac12de60 0x7f44ac12e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34290/0 (socket says 172.21.15.47:34290) 2024-09-15T21:57:03.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b1465640 1 -- 172.21.15.47:0/3244446287 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1833154661 0 0) 0x7f44ac099460 con 0x7f44ac12de60 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b1465640 1 -- 172.21.15.47:0/3244446287 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4498003520 con 0x7f44ac12de60 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b1c66640 1 -- 172.21.15.47:0/3244446287 >> v2:172.21.15.47:3300/0 conn(0x7f44ac06e560 msgr2=0x7f44ac12d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b1c66640 1 --2- 172.21.15.47:0/3244446287 >> v2:172.21.15.47:3300/0 conn(0x7f44ac06e560 0x7f44ac12d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b1c66640 1 -- 172.21.15.47:0/3244446287 >> v1:172.21.15.47:6789/0 conn(0x7f44ac12de60 legacy=0x7f44ac12e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b1c66640 1 -- 172.21.15.47:0/3244446287 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac06eed0 con 0x7f44ac12da00 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.228+0000 7f44b2467640 1 --2- 172.21.15.47:0/3244446287 >> v2:172.21.15.47:3300/0 conn(0x7f44ac06e560 0x7f44ac12d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.229+0000 7f44b1c66640 1 --2- 172.21.15.47:0/3244446287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12da00 0x7f44ac12e2d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f449c009870 tx=0x7f449c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa4c20024e665b1a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.229+0000 7f44b1465640 1 -- 172.21.15.47:0/3244446287 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c037070 con 0x7f44ac12da00 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.229+0000 7f44b1465640 1 -- 172.21.15.47:0/3244446287 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449c032440 con 0x7f44ac12da00 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.229+0000 7f44b1465640 1 -- 172.21.15.47:0/3244446287 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c032810 con 0x7f44ac12da00 2024-09-15T21:57:03.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.230+0000 7f44b46cb640 1 -- 172.21.15.47:0/3244446287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12da00 msgr2=0x7f44ac12e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.230+0000 7f44b46cb640 1 --2- 172.21.15.47:0/3244446287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12da00 0x7f44ac12e2d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f449c009870 tx=0x7f449c02a130 comp rx=0 tx=0).stop 2024-09-15T21:57:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.230+0000 7f44b46cb640 1 -- 172.21.15.47:0/3244446287 shutdown_connections 2024-09-15T21:57:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.230+0000 7f44b46cb640 1 --2- 172.21.15.47:0/3244446287 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12da00 0x7f44ac12e2d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.230+0000 7f44b46cb640 1 --2- 172.21.15.47:0/3244446287 >> v2:172.21.15.47:3300/0 conn(0x7f44ac06e560 0x7f44ac12d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.230+0000 7f44b46cb640 1 -- 172.21.15.47:0/3244446287 >> 172.21.15.47:0/3244446287 conn(0x7f44ac0a2160 msgr2=0x7f44ac0a2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.230+0000 7f44b46cb640 1 -- 172.21.15.47:0/3244446287 shutdown_connections 2024-09-15T21:57:03.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.230+0000 7f44b46cb640 1 -- 172.21.15.47:0/3244446287 wait complete. 2024-09-15T21:57:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b46cb640 1 Processor -- start 2024-09-15T21:57:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b46cb640 1 -- start start 2024-09-15T21:57:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b46cb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44ac06e560 0x7f44ac130ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b46cb640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f44ac12da00 0x7f44ac1313e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b46cb640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12de60 0x7f44ac131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b46cb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac137700 con 0x7f44ac06e560 2024-09-15T21:57:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b46cb640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac12f130 con 0x7f44ac12de60 2024-09-15T21:57:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b46cb640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f44ac137880 con 0x7f44ac12da00 2024-09-15T21:57:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b2467640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44ac06e560 0x7f44ac130ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b2467640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44ac06e560 0x7f44ac130ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58042/0 (socket says 172.21.15.47:58042) 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b2467640 1 -- 172.21.15.47:0/1414887221 learned_addr learned my addr 172.21.15.47:0/1414887221 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b1c66640 1 --2- 172.21.15.47:0/1414887221 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f44ac12da00 0x7f44ac1313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.231+0000 7f44b2c68640 1 --2- 172.21.15.47:0/1414887221 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12de60 0x7f44ac131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44b2467640 1 -- 172.21.15.47:0/1414887221 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f44ac12da00 msgr2=0x7f44ac1313e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44b2467640 1 --2- 172.21.15.47:0/1414887221 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f44ac12da00 0x7f44ac1313e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44b2467640 1 -- 172.21.15.47:0/1414887221 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12de60 msgr2=0x7f44ac131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44b2467640 1 --2- 172.21.15.47:0/1414887221 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f44ac12de60 0x7f44ac131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44b2467640 1 -- 172.21.15.47:0/1414887221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac132710 con 0x7f44ac06e560 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44b2467640 1 --2- 172.21.15.47:0/1414887221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44ac06e560 0x7f44ac130ea0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f44a800ebe0 tx=0x7f44a800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44a2ffd640 1 -- 172.21.15.47:0/1414887221 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8017070 con 0x7f44ac06e560 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44a2ffd640 1 -- 172.21.15.47:0/1414887221 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a800cc30 con 0x7f44ac06e560 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac131e60 con 0x7f44ac06e560 2024-09-15T21:57:03.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44a2ffd640 1 -- 172.21.15.47:0/1414887221 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8013470 con 0x7f44ac06e560 2024-09-15T21:57:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.232+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac133a40 con 0x7f44ac06e560 2024-09-15T21:57:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.233+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44ac12d4c0 con 0x7f44ac06e560 2024-09-15T21:57:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.233+0000 7f44a2ffd640 1 -- 172.21.15.47:0/1414887221 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f44a801a070 con 0x7f44ac06e560 2024-09-15T21:57:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.233+0000 7f44a2ffd640 1 -- 172.21.15.47:0/1414887221 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f44a805d940 con 0x7f44ac06e560 2024-09-15T21:57:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.233+0000 7f44a2ffd640 1 -- 172.21.15.47:0/1414887221 --> v1:172.21.15.47:6813/2978958959 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44a8016ec0 con 0x7f449004e880 2024-09-15T21:57:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.233+0000 7f44a2ffd640 1 -- 172.21.15.47:0/1414887221 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f44a80622c0 con 0x7f44ac06e560 2024-09-15T21:57:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.234+0000 7f44a2ffd640 1 -- 172.21.15.47:0/1414887221 <== osd.3 v1:172.21.15.47:6813/2978958959 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f44a8016ec0 con 0x7f449004e880 2024-09-15T21:57:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.247+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 --> v1:172.21.15.47:6813/2978958959 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f44ac062880 con 0x7f449004e880 2024-09-15T21:57:03.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.247+0000 7f44a2ffd640 1 -- 172.21.15.47:0/1414887221 <== osd.3 v1:172.21.15.47:6813/2978958959 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (unknown 0 0 125856537) 0x7f44ac062880 con 0x7f449004e880 2024-09-15T21:57:03.250 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-15T21:57:03.250 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-15T21:57:03.250 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-15T21:57:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.247+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 >> v1:172.21.15.47:6813/2978958959 conn(0x7f449004e880 legacy=0x7f4490050cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.247+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 >> v1:172.21.15.47:6800/1790418198 conn(0x7f4490048000 legacy=0x7f449004a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.247+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44ac06e560 msgr2=0x7f44ac130ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.247+0000 7f44b46cb640 1 --2- 172.21.15.47:0/1414887221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44ac06e560 0x7f44ac130ea0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f44a800ebe0 tx=0x7f44a800c520 comp rx=0 tx=0).stop 2024-09-15T21:57:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.247+0000 7f44b2c68640 1 -- 172.21.15.47:0/1414887221 reap_dead start 2024-09-15T21:57:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.247+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 shutdown_connections 2024-09-15T21:57:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.247+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 >> 172.21.15.47:0/1414887221 conn(0x7f44ac0a2160 msgr2=0x7f44ac09a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.248+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 shutdown_connections 2024-09-15T21:57:03.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.248+0000 7f44b46cb640 1 -- 172.21.15.47:0/1414887221 wait complete. 2024-09-15T21:57:03.260 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T21:57:03.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.359+0000 7f7e29867640 1 Processor -- start 2024-09-15T21:57:03.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e29867640 1 -- start start 2024-09-15T21:57:03.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e29867640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24097890 0x7f7e24097c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e29867640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24134db0 0x7f7e241371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e29867640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e24098f70 con 0x7f7e2409b620 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e29867640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f7e24096f00 con 0x7f7e24134db0 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e29867640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f7e240994c0 con 0x7f7e24097890 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e22ffd640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7e2409b620 0x7f7e24097180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34292/0 (socket says 172.21.15.47:34292) 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e22ffd640 1 -- 172.21.15.47:0/839271288 learned_addr learned my addr 172.21.15.47:0/839271288 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e237fe640 1 --2- 172.21.15.47:0/839271288 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24134db0 0x7f7e241371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e227fc640 1 --2- 172.21.15.47:0/839271288 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24097890 0x7f7e24097c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e21ffb640 1 -- 172.21.15.47:0/839271288 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1887873584 0 0) 0x7f7e24098f70 con 0x7f7e2409b620 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e21ffb640 1 -- 172.21.15.47:0/839271288 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e0c003520 con 0x7f7e2409b620 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e227fc640 1 -- 172.21.15.47:0/839271288 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24134db0 msgr2=0x7f7e241371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:03.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e227fc640 1 --2- 172.21.15.47:0/839271288 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24134db0 0x7f7e241371a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e227fc640 1 -- 172.21.15.47:0/839271288 >> v1:172.21.15.47:6789/0 conn(0x7f7e2409b620 legacy=0x7f7e24097180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.360+0000 7f7e227fc640 1 -- 172.21.15.47:0/839271288 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2409c3b0 con 0x7f7e24097890 2024-09-15T21:57:03.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.361+0000 7f7e237fe640 1 --2- 172.21.15.47:0/839271288 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24134db0 0x7f7e241371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:03.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.361+0000 7f7e227fc640 1 --2- 172.21.15.47:0/839271288 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24097890 0x7f7e24097c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e180097d0 tx=0x7f7e1802a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb300e29935f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:03.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.361+0000 7f7e21ffb640 1 -- 172.21.15.47:0/839271288 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18037070 con 0x7f7e24097890 2024-09-15T21:57:03.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.361+0000 7f7e21ffb640 1 -- 172.21.15.47:0/839271288 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1802ad90 con 0x7f7e24097890 2024-09-15T21:57:03.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.361+0000 7f7e21ffb640 1 -- 172.21.15.47:0/839271288 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18032660 con 0x7f7e24097890 2024-09-15T21:57:03.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.362+0000 7f7e29867640 1 -- 172.21.15.47:0/839271288 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24097890 msgr2=0x7f7e24097c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.362+0000 7f7e29867640 1 --2- 172.21.15.47:0/839271288 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24097890 0x7f7e24097c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7e180097d0 tx=0x7f7e1802a040 comp rx=0 tx=0).stop 2024-09-15T21:57:03.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.362+0000 7f7e29867640 1 -- 172.21.15.47:0/839271288 shutdown_connections 2024-09-15T21:57:03.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.362+0000 7f7e29867640 1 --2- 172.21.15.47:0/839271288 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24134db0 0x7f7e241371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.362+0000 7f7e29867640 1 --2- 172.21.15.47:0/839271288 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24097890 0x7f7e24097c70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.362+0000 7f7e29867640 1 -- 172.21.15.47:0/839271288 >> 172.21.15.47:0/839271288 conn(0x7f7e2409c8f0 msgr2=0x7f7e2409cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:03.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.362+0000 7f7e29867640 1 -- 172.21.15.47:0/839271288 shutdown_connections 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.362+0000 7f7e29867640 1 -- 172.21.15.47:0/839271288 wait complete. 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.363+0000 7f7e29867640 1 Processor -- start 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.363+0000 7f7e29867640 1 -- start start 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.363+0000 7f7e29867640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24097890 0x7f7e24092b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.363+0000 7f7e29867640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2409b620 0x7f7e240930d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.363+0000 7f7e29867640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24134db0 0x7f7e24093610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.363+0000 7f7e29867640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24137860 con 0x7f7e2409b620 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.363+0000 7f7e29867640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f7e241376e0 con 0x7f7e24097890 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.363+0000 7f7e29867640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f7e241379e0 con 0x7f7e24134db0 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e227fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2409b620 0x7f7e240930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.363+0000 7f7e22ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24097890 0x7f7e24092b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e227fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2409b620 0x7f7e240930d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58050/0 (socket says 172.21.15.47:58050) 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e237fe640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24134db0 0x7f7e24093610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e227fc640 1 -- 172.21.15.47:0/2625675003 learned_addr learned my addr 172.21.15.47:0/2625675003 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e22ffd640 1 --2- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24097890 0x7f7e24092b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45508/0 (socket says 172.21.15.47:45508) 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e22ffd640 1 -- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24134db0 msgr2=0x7f7e24093610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e22ffd640 1 --2- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24134db0 0x7f7e24093610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e22ffd640 1 -- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2409b620 msgr2=0x7f7e240930d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e22ffd640 1 --2- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2409b620 0x7f7e240930d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e22ffd640 1 -- 172.21.15.47:0/2625675003 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e24095ec0 con 0x7f7e24097890 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e227fc640 1 --2- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2409b620 0x7f7e240930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e22ffd640 1 --2- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24097890 0x7f7e24092b90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400c430 tx=0x7f7e1400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:03.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e0bfff640 1 -- 172.21.15.47:0/2625675003 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14017070 con 0x7f7e24097890 2024-09-15T21:57:03.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e0bfff640 1 -- 172.21.15.47:0/2625675003 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e14005ce0 con 0x7f7e24097890 2024-09-15T21:57:03.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e29867640 1 -- 172.21.15.47:0/2625675003 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e24094dc0 con 0x7f7e24097890 2024-09-15T21:57:03.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e0bfff640 1 -- 172.21.15.47:0/2625675003 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e140072d0 con 0x7f7e24097890 2024-09-15T21:57:03.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.364+0000 7f7e29867640 1 -- 172.21.15.47:0/2625675003 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e24073a40 con 0x7f7e24097890 2024-09-15T21:57:03.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.365+0000 7f7e29867640 1 -- 172.21.15.47:0/2625675003 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e24095620 con 0x7f7e24097890 2024-09-15T21:57:03.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.365+0000 7f7e0bfff640 1 -- 172.21.15.47:0/2625675003 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f7e1401a070 con 0x7f7e24097890 2024-09-15T21:57:03.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.366+0000 7f7e0bfff640 1 -- 172.21.15.47:0/2625675003 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f7e1405cd50 con 0x7f7e24097890 2024-09-15T21:57:03.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.369+0000 7f7e0bfff640 1 -- 172.21.15.47:0/2625675003 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f7e24095620 con 0x7f7e24097890 2024-09-15T21:57:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.517+0000 7f7e29867640 1 -- 172.21.15.47:0/2625675003 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f7e24097c70 con 0x7f7e10047fd0 2024-09-15T21:57:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.517+0000 7f7e0bfff640 1 -- 172.21.15.47:0/2625675003 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (unknown 3526570807 0 0) 0x7f7e24097c70 con 0x7f7e10047fd0 2024-09-15T21:57:03.520 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T21:57:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.519+0000 7f7e29867640 1 -- 172.21.15.47:0/2625675003 >> v1:172.21.15.47:6800/1790418198 conn(0x7f7e10047fd0 legacy=0x7f7e1004a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.519+0000 7f7e29867640 1 -- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24097890 msgr2=0x7f7e24092b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.519+0000 7f7e29867640 1 --2- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24097890 0x7f7e24092b90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400c430 tx=0x7f7e1400c8f0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.519+0000 7f7e29867640 1 -- 172.21.15.47:0/2625675003 shutdown_connections 2024-09-15T21:57:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.519+0000 7f7e29867640 1 --2- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f7e24134db0 0x7f7e24093610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.519+0000 7f7e29867640 1 --2- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7e2409b620 0x7f7e240930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.519+0000 7f7e29867640 1 --2- 172.21.15.47:0/2625675003 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f7e24097890 0x7f7e24092b90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.519+0000 7f7e29867640 1 -- 172.21.15.47:0/2625675003 >> 172.21.15.47:0/2625675003 conn(0x7f7e2409c8f0 msgr2=0x7f7e2409e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.519+0000 7f7e29867640 1 -- 172.21.15.47:0/2625675003 shutdown_connections 2024-09-15T21:57:03.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.519+0000 7f7e29867640 1 -- 172.21.15.47:0/2625675003 wait complete. 2024-09-15T21:57:03.533 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.631+0000 7f158d48f640 1 Processor -- start 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.631+0000 7f158d48f640 1 -- start start 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f158d48f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 0x7f158812f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f158d48f640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15880987a0 0x7f1588137930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f158d48f640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f158809f230 con 0x7f158812fce0 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f158d48f640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f158806f260 con 0x7f15880987a0 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f158d48f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f1588099600 con 0x7f158812f3c0 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f1586ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 0x7f158812f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f15877fe640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f15880987a0 0x7f1588137930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f1586ffd640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 0x7f158812f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:45522/0 (socket says 172.21.15.47:45522) 2024-09-15T21:57:03.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f1586ffd640 1 -- 172.21.15.47:0/1038061080 learned_addr learned my addr 172.21.15.47:0/1038061080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f15867fc640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f158812fce0 0x7f1588098090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34302/0 (socket says 172.21.15.47:34302) 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f1586ffd640 1 -- 172.21.15.47:0/1038061080 >> v2:172.21.15.47:3300/0 conn(0x7f15880987a0 msgr2=0x7f1588137930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f1586ffd640 1 --2- 172.21.15.47:0/1038061080 >> v2:172.21.15.47:3300/0 conn(0x7f15880987a0 0x7f1588137930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f1586ffd640 1 -- 172.21.15.47:0/1038061080 >> v1:172.21.15.47:6789/0 conn(0x7f158812fce0 legacy=0x7f1588098090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f1586ffd640 1 -- 172.21.15.47:0/1038061080 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1588130120 con 0x7f158812f3c0 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.632+0000 7f15877fe640 1 --2- 172.21.15.47:0/1038061080 >> v2:172.21.15.47:3300/0 conn(0x7f15880987a0 0x7f1588137930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.633+0000 7f1586ffd640 1 --2- 172.21.15.47:0/1038061080 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 0x7f158812f7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1588099780 tx=0x7f157802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=250eb339bbf6e46e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.633+0000 7f1585ffb640 1 -- 172.21.15.47:0/1038061080 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578037070 con 0x7f158812f3c0 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.633+0000 7f1585ffb640 1 -- 172.21.15.47:0/1038061080 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1578032440 con 0x7f158812f3c0 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.633+0000 7f1585ffb640 1 -- 172.21.15.47:0/1038061080 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578032810 con 0x7f158812f3c0 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.634+0000 7f158d48f640 1 -- 172.21.15.47:0/1038061080 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 msgr2=0x7f158812f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.634+0000 7f158d48f640 1 --2- 172.21.15.47:0/1038061080 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 0x7f158812f7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1588099780 tx=0x7f157802a130 comp rx=0 tx=0).stop 2024-09-15T21:57:03.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.634+0000 7f158d48f640 1 -- 172.21.15.47:0/1038061080 shutdown_connections 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.634+0000 7f158d48f640 1 --2- 172.21.15.47:0/1038061080 >> v2:172.21.15.47:3300/0 conn(0x7f15880987a0 0x7f1588137930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.634+0000 7f158d48f640 1 --2- 172.21.15.47:0/1038061080 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 0x7f158812f7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.634+0000 7f158d48f640 1 -- 172.21.15.47:0/1038061080 >> 172.21.15.47:0/1038061080 conn(0x7f15880a23b0 msgr2=0x7f158809e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.634+0000 7f158d48f640 1 -- 172.21.15.47:0/1038061080 shutdown_connections 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.634+0000 7f158d48f640 1 -- 172.21.15.47:0/1038061080 wait complete. 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f158d48f640 1 Processor -- start 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f158d48f640 1 -- start start 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f158d48f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15880987a0 0x7f15881c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f158d48f640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 0x7f15881c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f158d48f640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f158812fce0 0x7f158808cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f158d48f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1588098c10 con 0x7f15880987a0 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f158d48f640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f158806ee60 con 0x7f158812f3c0 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f158d48f640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f1588098d90 con 0x7f158812fce0 2024-09-15T21:57:03.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f1586ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15880987a0 0x7f15881c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f1586ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15880987a0 0x7f15881c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58064/0 (socket says 172.21.15.47:58064) 2024-09-15T21:57:03.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f1586ffd640 1 -- 172.21.15.47:0/4146186851 learned_addr learned my addr 172.21.15.47:0/4146186851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:03.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f15877fe640 1 --2- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f158812fce0 0x7f158808cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.635+0000 7f15867fc640 1 --2- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 0x7f15881c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:03.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f1586ffd640 1 -- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f158812fce0 msgr2=0x7f158808cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f1586ffd640 1 --2- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f158812fce0 0x7f158808cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f1586ffd640 1 -- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 msgr2=0x7f15881c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f1586ffd640 1 --2- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 0x7f15881c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f1586ffd640 1 -- 172.21.15.47:0/4146186851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1588090500 con 0x7f15880987a0 2024-09-15T21:57:03.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f1586ffd640 1 --2- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15880987a0 0x7f15881c4960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f157802a170 tx=0x7f15780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:03.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f156f7fe640 1 -- 172.21.15.47:0/4146186851 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578037040 con 0x7f15880987a0 2024-09-15T21:57:03.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f156f7fe640 1 -- 172.21.15.47:0/4146186851 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1578042070 con 0x7f15880987a0 2024-09-15T21:57:03.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f158d48f640 1 -- 172.21.15.47:0/4146186851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158808f1a0 con 0x7f15880987a0 2024-09-15T21:57:03.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f156f7fe640 1 -- 172.21.15.47:0/4146186851 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578030900 con 0x7f15880987a0 2024-09-15T21:57:03.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.636+0000 7f158d48f640 1 -- 172.21.15.47:0/4146186851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158808fb30 con 0x7f15880987a0 2024-09-15T21:57:03.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.637+0000 7f158d48f640 1 -- 172.21.15.47:0/4146186851 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158812f7a0 con 0x7f15880987a0 2024-09-15T21:57:03.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.637+0000 7f156f7fe640 1 -- 172.21.15.47:0/4146186851 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f1578030aa0 con 0x7f15880987a0 2024-09-15T21:57:03.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.637+0000 7f156f7fe640 1 -- 172.21.15.47:0/4146186851 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f157807e240 con 0x7f15880987a0 2024-09-15T21:57:03.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.641+0000 7f156f7fe640 1 -- 172.21.15.47:0/4146186851 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f157807d2f0 con 0x7f15880987a0 2024-09-15T21:57:03.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.789+0000 7f158d48f640 1 -- 172.21.15.47:0/4146186851 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1588068e40 con 0x7f1574048000 2024-09-15T21:57:03.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.790+0000 7f156f7fe640 1 -- 172.21.15.47:0/4146186851 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32004 (unknown 2965378022 0 3468866284) 0x7f1588068e40 con 0x7f1574048000 2024-09-15T21:57:03.793 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:57:03.793 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:57:03.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.791+0000 7f158d48f640 1 -- 172.21.15.47:0/4146186851 >> v1:172.21.15.47:6800/1790418198 conn(0x7f1574048000 legacy=0x7f157404a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.791+0000 7f158d48f640 1 -- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15880987a0 msgr2=0x7f15881c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:03.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.791+0000 7f158d48f640 1 --2- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15880987a0 0x7f15881c4960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f157802a170 tx=0x7f15780046c0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.791+0000 7f158d48f640 1 -- 172.21.15.47:0/4146186851 shutdown_connections 2024-09-15T21:57:03.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.791+0000 7f158d48f640 1 --2- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f158812fce0 0x7f158808cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.791+0000 7f158d48f640 1 --2- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f158812f3c0 0x7f15881c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.791+0000 7f158d48f640 1 --2- 172.21.15.47:0/4146186851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15880987a0 0x7f15881c4960 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:03.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.791+0000 7f158d48f640 1 -- 172.21.15.47:0/4146186851 >> 172.21.15.47:0/4146186851 conn(0x7f15880a23b0 msgr2=0x7f1588099b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:03.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.792+0000 7f158d48f640 1 -- 172.21.15.47:0/4146186851 shutdown_connections 2024-09-15T21:57:03.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:03.792+0000 7f158d48f640 1 -- 172.21.15.47:0/4146186851 wait complete. 2024-09-15T21:57:03.805 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-15T21:57:03.677170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128968,"kb_used_data":2192,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358392,"statfs":{"total":386547056640,"available":386414993408,"internally_reserved":0,"allocated":2244608,"data_stored":1543286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.936573"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185545+0000","last_change":"2024-09-15T21:56:49.850959+0000","last_active":"2024-09-15T21:56:56.185545+0000","last_peered":"2024-09-15T21:56:56.185545+0000","last_clean":"2024-09-15T21:56:56.185545+0000","last_became_active":"2024-09-15T21:56:49.850607+0000","last_became_peered":"2024-09-15T21:56:49.850607+0000","last_unstale":"2024-09-15T21:56:56.185545+0000","last_undegraded":"2024-09-15T21:56:56.185545+0000","last_fullsized":"2024-09-15T21:56:56.185545+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:03.532841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743750+0000","last_change":"2024-09-15T21:56:47.710370+0000","last_active":"2024-09-15T21:56:54.743750+0000","last_peered":"2024-09-15T21:56:54.743750+0000","last_clean":"2024-09-15T21:56:54.743750+0000","last_became_active":"2024-09-15T21:56:47.710067+0000","last_became_peered":"2024-09-15T21:56:47.710067+0000","last_unstale":"2024-09-15T21:56:54.743750+0000","last_undegraded":"2024-09-15T21:56:54.743750+0000","last_fullsized":"2024-09-15T21:56:54.743750+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:50.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185549+0000","last_change":"2024-09-15T21:56:49.851508+0000","last_active":"2024-09-15T21:56:56.185549+0000","last_peered":"2024-09-15T21:56:56.185549+0000","last_clean":"2024-09-15T21:56:56.185549+0000","last_became_active":"2024-09-15T21:56:49.851081+0000","last_became_peered":"2024-09-15T21:56:49.851081+0000","last_unstale":"2024-09-15T21:56:56.185549+0000","last_undegraded":"2024-09-15T21:56:56.185549+0000","last_fullsized":"2024-09-15T21:56:56.185549+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:22:03.698746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743740+0000","last_change":"2024-09-15T21:56:47.711356+0000","last_active":"2024-09-15T21:56:54.743740+0000","last_peered":"2024-09-15T21:56:54.743740+0000","last_clean":"2024-09-15T21:56:54.743740+0000","last_became_active":"2024-09-15T21:56:47.711012+0000","last_became_peered":"2024-09-15T21:56:47.711012+0000","last_unstale":"2024-09-15T21:56:54.743740+0000","last_undegraded":"2024-09-15T21:56:54.743740+0000","last_fullsized":"2024-09-15T21:56:54.743740+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:58:23.626385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185508+0000","last_change":"2024-09-15T21:56:49.733169+0000","last_active":"2024-09-15T21:56:56.185508+0000","last_peered":"2024-09-15T21:56:56.185508+0000","last_clean":"2024-09-15T21:56:56.185508+0000","last_became_active":"2024-09-15T21:56:49.732754+0000","last_became_peered":"2024-09-15T21:56:49.732754+0000","last_unstale":"2024-09-15T21:56:56.185508+0000","last_undegraded":"2024-09-15T21:56:56.185508+0000","last_fullsized":"2024-09-15T21:56:56.185508+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:11.761893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743733+0000","last_change":"2024-09-15T21:56:49.733734+0000","last_active":"2024-09-15T21:56:54.743733+0000","last_peered":"2024-09-15T21:56:54.743733+0000","last_clean":"2024-09-15T21:56:54.743733+0000","last_became_active":"2024-09-15T21:56:49.733153+0000","last_became_peered":"2024-09-15T21:56:49.733153+0000","last_unstale":"2024-09-15T21:56:54.743733+0000","last_undegraded":"2024-09-15T21:56:54.743733+0000","last_fullsized":"2024-09-15T21:56:54.743733+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:56.860565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185503+0000","last_change":"2024-09-15T21:56:49.731747+0000","last_active":"2024-09-15T21:56:56.185503+0000","last_peered":"2024-09-15T21:56:56.185503+0000","last_clean":"2024-09-15T21:56:56.185503+0000","last_became_active":"2024-09-15T21:56:49.731371+0000","last_became_peered":"2024-09-15T21:56:49.731371+0000","last_unstale":"2024-09-15T21:56:56.185503+0000","last_undegraded":"2024-09-15T21:56:56.185503+0000","last_fullsized":"2024-09-15T21:56:56.185503+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:27.373825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:56.185514+0000","last_change":"2024-09-15T21:56:49.733242+0000","last_active":"2024-09-15T21:56:56.185514+0000","last_peered":"2024-09-15T21:56:56.185514+0000","last_clean":"2024-09-15T21:56:56.185514+0000","last_became_active":"2024-09-15T21:56:49.732855+0000","last_became_peered":"2024-09-15T21:56:49.732855+0000","last_unstale":"2024-09-15T21:56:56.185514+0000","last_undegraded":"2024-09-15T21:56:56.185514+0000","last_fullsized":"2024-09-15T21:56:56.185514+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:17.263879+0000","last_clean_scrub_stamp":"2024-09-15T21:55:17.263879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:50:43.371057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":146,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:56:54.743991+0000","last_change":"2024-09-15T21:56:47.710563+0000","last_active":"2024-09-15T21:56:54.743991+0000","last_peered":"2024-09-15T21:56:54.743991+0000","last_clean":"2024-09-15T21:56:54.743991+0000","last_became_active":"2024-09-15T21:56:47.710235+0000","last_became_peered":"2024-09-15T21:56:47.710235+0000","last_unstale":"2024-09-15T21:56:54.743991+0000","last_undegraded":"2024-09-15T21:56:54.743991+0000","last_fullsized":"2024-09-15T21:56:54.743991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:55:11.249936+0000","last_clean_scrub_stamp":"2024-09-15T21:55:11.249936+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:46:43.977752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":27,"seq":115964116997,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":266240,"data_stored":90647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":27,"seq":115964116997,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":262144,"data_stored":90628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":25,"seq":107374182405,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":840,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":860160,"data_stored":681015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182405,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":836,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":856064,"data_stored":680996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:57:03.808 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T21:55:17.263879+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=55, tm_sec=17, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=57, tm_sec=0, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:57:03.808 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T21:55:17.263879+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=55, tm_sec=17, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=57, tm_sec=0, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:57:03.808 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T21:55:17.263879+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=55, tm_sec=17, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=57, tm_sec=0, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:57:03.808 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T21:55:17.263879+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=55, tm_sec=17, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=57, tm_sec=0, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:57:03.809 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T21:55:17.263879+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=55, tm_sec=17, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=57, tm_sec=0, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:57:03.809 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T21:55:17.263879+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=55, tm_sec=17, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=57, tm_sec=0, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:57:03.809 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T21:55:17.263879+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=55, tm_sec=17, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=57, tm_sec=0, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:57:03.809 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T21:55:17.263879+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=55, tm_sec=17, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=57, tm_sec=0, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:57:03.809 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T21:55:11.249936+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=55, tm_sec=11, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=57, tm_sec=0, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:57:03.809 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T21:57:23.810 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:57:23.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.923+0000 7f3b50cf0640 1 Processor -- start 2024-09-15T21:57:23.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.923+0000 7f3b50cf0640 1 -- start start 2024-09-15T21:57:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b50cf0640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b4c12f610 0x7f3b4c12f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:23.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b50cf0640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12ff30 0x7f3b4c12c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b50cf0640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c12cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b50cf0640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3b4c06f350 con 0x7f3b4c12f610 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b50cf0640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c0a0d60 con 0x7f3b4c12c650 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b50cf0640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f3b4c099c20 con 0x7f3b4c12ff30 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b4a59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b4c12f610 0x7f3b4c12f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b49d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12ff30 0x7f3b4c12c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b4a59c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3b4c12f610 0x7f3b4c12f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57758/0 (socket says 172.21.15.47:57758) 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b4a59c640 1 -- 172.21.15.47:0/3878599219 learned_addr learned my addr 172.21.15.47:0/3878599219 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b49d9b640 1 --2- 172.21.15.47:0/3878599219 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12ff30 0x7f3b4c12c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:59806/0 (socket says 172.21.15.47:59806) 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b4ad9d640 1 --2- 172.21.15.47:0/3878599219 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c12cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:23.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b49d9b640 1 -- 172.21.15.47:0/3878599219 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 msgr2=0x7f3b4c12cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b49d9b640 1 --2- 172.21.15.47:0/3878599219 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c12cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b49d9b640 1 -- 172.21.15.47:0/3878599219 >> v2:172.21.15.47:3300/0 conn(0x7f3b4c12f610 msgr2=0x7f3b4c12f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b49d9b640 1 --2- 172.21.15.47:0/3878599219 >> v2:172.21.15.47:3300/0 conn(0x7f3b4c12f610 0x7f3b4c12f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b49d9b640 1 -- 172.21.15.47:0/3878599219 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c0998e0 con 0x7f3b4c12ff30 2024-09-15T21:57:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b4a59c640 1 --2- 172.21.15.47:0/3878599219 >> v2:172.21.15.47:3300/0 conn(0x7f3b4c12f610 0x7f3b4c12f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.924+0000 7f3b4ad9d640 1 --2- 172.21.15.47:0/3878599219 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c12cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.925+0000 7f3b49d9b640 1 --2- 172.21.15.47:0/3878599219 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12ff30 0x7f3b4c12c110 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b400097d0 tx=0x7f3b4002a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61d7a1091fbdd435 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.925+0000 7f3b4959a640 1 -- 172.21.15.47:0/3878599219 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40037070 con 0x7f3b4c12ff30 2024-09-15T21:57:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.925+0000 7f3b4959a640 1 -- 172.21.15.47:0/3878599219 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4002ad10 con 0x7f3b4c12ff30 2024-09-15T21:57:23.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.925+0000 7f3b4959a640 1 -- 172.21.15.47:0/3878599219 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40032660 con 0x7f3b4c12ff30 2024-09-15T21:57:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.926+0000 7f3b50cf0640 1 -- 172.21.15.47:0/3878599219 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12ff30 msgr2=0x7f3b4c12c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.926+0000 7f3b50cf0640 1 --2- 172.21.15.47:0/3878599219 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12ff30 0x7f3b4c12c110 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b400097d0 tx=0x7f3b4002a000 comp rx=0 tx=0).stop 2024-09-15T21:57:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.926+0000 7f3b50cf0640 1 -- 172.21.15.47:0/3878599219 shutdown_connections 2024-09-15T21:57:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.926+0000 7f3b50cf0640 1 --2- 172.21.15.47:0/3878599219 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c12cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.926+0000 7f3b50cf0640 1 --2- 172.21.15.47:0/3878599219 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12ff30 0x7f3b4c12c110 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.926+0000 7f3b50cf0640 1 --2- 172.21.15.47:0/3878599219 >> v2:172.21.15.47:3300/0 conn(0x7f3b4c12f610 0x7f3b4c12f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.926+0000 7f3b50cf0640 1 -- 172.21.15.47:0/3878599219 >> 172.21.15.47:0/3878599219 conn(0x7f3b4c09ec70 msgr2=0x7f3b4c09f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.926+0000 7f3b50cf0640 1 -- 172.21.15.47:0/3878599219 shutdown_connections 2024-09-15T21:57:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.926+0000 7f3b50cf0640 1 -- 172.21.15.47:0/3878599219 wait complete. 2024-09-15T21:57:23.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.927+0000 7f3b50cf0640 1 Processor -- start 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.927+0000 7f3b50cf0640 1 -- start start 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.927+0000 7f3b50cf0640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c131020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.927+0000 7f3b50cf0640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12f610 0x7f3b4c131560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.927+0000 7f3b50cf0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b4c12ff30 0x7f3b4c131aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b50cf0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c136940 con 0x7f3b4c12ff30 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b50cf0640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c130580 con 0x7f3b4c12c650 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b50cf0640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f3b4c136ac0 con 0x7f3b4c12f610 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b4a59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c131020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b49d9b640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12f610 0x7f3b4c131560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b4ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b4c12ff30 0x7f3b4c131aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b4a59c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c131020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3301/0 says I am v2:172.21.15.47:36944/0 (socket says 172.21.15.47:36944) 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b4a59c640 1 -- 172.21.15.47:0/962532433 learned_addr learned my addr 172.21.15.47:0/962532433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:23.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b49d9b640 1 -- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 msgr2=0x7f3b4c131020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b49d9b640 1 --2- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c131020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b49d9b640 1 -- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b4c12ff30 msgr2=0x7f3b4c131aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:23.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b49d9b640 1 --2- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b4c12ff30 0x7f3b4c131aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b49d9b640 1 -- 172.21.15.47:0/962532433 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c136590 con 0x7f3b4c12f610 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b4a59c640 1 --2- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c131020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b4ad9d640 1 --2- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b4c12ff30 0x7f3b4c131aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b49d9b640 1 --2- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12f610 0x7f3b4c131560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c069a30 tx=0x7f3b400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b3affd640 1 -- 172.21.15.47:0/962532433 <== mon.2 v2:172.21.15.47:3302/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40041020 con 0x7f3b4c12f610 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b3affd640 1 -- 172.21.15.47:0/962532433 <== mon.2 v2:172.21.15.47:3302/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4003d680 con 0x7f3b4c12f610 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b50cf0640 1 -- 172.21.15.47:0/962532433 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c133eb0 con 0x7f3b4c12f610 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b3affd640 1 -- 172.21.15.47:0/962532433 <== mon.2 v2:172.21.15.47:3302/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40037040 con 0x7f3b4c12f610 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.928+0000 7f3b50cf0640 1 -- 172.21.15.47:0/962532433 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c134ae0 con 0x7f3b4c12f610 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.929+0000 7f3b3affd640 1 -- 172.21.15.47:0/962532433 <== mon.2 v2:172.21.15.47:3302/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f3b4002abd0 con 0x7f3b4c12f610 2024-09-15T21:57:23.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.929+0000 7f3b50cf0640 1 -- 172.21.15.47:0/962532433 --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c12f9f0 con 0x7f3b4c12f610 2024-09-15T21:57:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.930+0000 7f3b3affd640 1 -- 172.21.15.47:0/962532433 <== mon.2 v2:172.21.15.47:3302/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f3b4007d9e0 con 0x7f3b4c12f610 2024-09-15T21:57:23.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:23.933+0000 7f3b3affd640 1 -- 172.21.15.47:0/962532433 <== mon.2 v2:172.21.15.47:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f3b4004b020 con 0x7f3b4c12f610 2024-09-15T21:57:24.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.077+0000 7f3b50cf0640 1 -- 172.21.15.47:0/962532433 --> v1:172.21.15.47:6800/1790418198 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b4c132b90 con 0x7f3b2c048000 2024-09-15T21:57:24.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.078+0000 7f3b3affd640 1 -- 172.21.15.47:0/962532433 <== mgr.14098 v1:172.21.15.47:6800/1790418198 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32013 (unknown 2965378022 0 3166672186) 0x7f3b4c132b90 con 0x7f3b2c048000 2024-09-15T21:57:24.080 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-15T21:57:24.081 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-09-15T21:57:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.080+0000 7f3b50cf0640 1 -- 172.21.15.47:0/962532433 >> v1:172.21.15.47:6800/1790418198 conn(0x7f3b2c048000 legacy=0x7f3b2c04a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.080+0000 7f3b50cf0640 1 -- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12f610 msgr2=0x7f3b4c131560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.080+0000 7f3b50cf0640 1 --2- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12f610 0x7f3b4c131560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c069a30 tx=0x7f3b400046c0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.080+0000 7f3b50cf0640 1 -- 172.21.15.47:0/962532433 shutdown_connections 2024-09-15T21:57:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.080+0000 7f3b50cf0640 1 --2- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b4c12ff30 0x7f3b4c131aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.080+0000 7f3b50cf0640 1 --2- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f3b4c12f610 0x7f3b4c131560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.080+0000 7f3b50cf0640 1 --2- 172.21.15.47:0/962532433 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f3b4c12c650 0x7f3b4c131020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.080+0000 7f3b50cf0640 1 -- 172.21.15.47:0/962532433 >> 172.21.15.47:0/962532433 conn(0x7f3b4c09ec70 msgr2=0x7f3b4c09d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.080+0000 7f3b50cf0640 1 -- 172.21.15.47:0/962532433 shutdown_connections 2024-09-15T21:57:24.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.080+0000 7f3b50cf0640 1 -- 172.21.15.47:0/962532433 wait complete. 2024-09-15T21:57:24.092 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-15T21:57:23.681092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128944,"kb_used_data":2192,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358416,"statfs":{"total":386547056640,"available":386415017984,"internally_reserved":0,"allocated":2244608,"data_stored":1543286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002377"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:57:07.416478+0000","last_change":"2024-09-15T21:57:07.416503+0000","last_active":"2024-09-15T21:57:07.416478+0000","last_peered":"2024-09-15T21:57:07.416478+0000","last_clean":"2024-09-15T21:57:07.416478+0000","last_became_active":"2024-09-15T21:56:49.850607+0000","last_became_peered":"2024-09-15T21:56:49.850607+0000","last_unstale":"2024-09-15T21:57:07.416478+0000","last_undegraded":"2024-09-15T21:57:07.416478+0000","last_fullsized":"2024-09-15T21:57:07.416478+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:57:07.416383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:57:07.416383+0000","last_clean_scrub_stamp":"2024-09-15T21:57:07.416383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:34.302299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035164899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:57:03.419948+0000","last_change":"2024-09-15T21:57:03.419986+0000","last_active":"2024-09-15T21:57:03.419948+0000","last_peered":"2024-09-15T21:57:03.419948+0000","last_clean":"2024-09-15T21:57:03.419948+0000","last_became_active":"2024-09-15T21:56:47.710067+0000","last_became_peered":"2024-09-15T21:56:47.710067+0000","last_unstale":"2024-09-15T21:57:03.419948+0000","last_undegraded":"2024-09-15T21:57:03.419948+0000","last_fullsized":"2024-09-15T21:57:03.419948+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:57:03.419797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:57:03.419797+0000","last_clean_scrub_stamp":"2024-09-15T21:57:03.419797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:15:24.998167+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00083232599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:57:08.446280+0000","last_change":"2024-09-15T21:57:08.446332+0000","last_active":"2024-09-15T21:57:08.446280+0000","last_peered":"2024-09-15T21:57:08.446280+0000","last_clean":"2024-09-15T21:57:08.446280+0000","last_became_active":"2024-09-15T21:56:49.851081+0000","last_became_peered":"2024-09-15T21:56:49.851081+0000","last_unstale":"2024-09-15T21:57:08.446280+0000","last_undegraded":"2024-09-15T21:57:08.446280+0000","last_fullsized":"2024-09-15T21:57:08.446280+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:57:08.446168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:57:08.446168+0000","last_clean_scrub_stamp":"2024-09-15T21:57:08.446168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:46:42.082610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095906400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:57:05.428860+0000","last_change":"2024-09-15T21:57:05.428893+0000","last_active":"2024-09-15T21:57:05.428860+0000","last_peered":"2024-09-15T21:57:05.428860+0000","last_clean":"2024-09-15T21:57:05.428860+0000","last_became_active":"2024-09-15T21:56:47.711012+0000","last_became_peered":"2024-09-15T21:56:47.711012+0000","last_unstale":"2024-09-15T21:57:05.428860+0000","last_undegraded":"2024-09-15T21:57:05.428860+0000","last_fullsized":"2024-09-15T21:57:05.428860+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:57:05.428766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:57:05.428766+0000","last_clean_scrub_stamp":"2024-09-15T21:57:05.428766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:14.349627+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00049360799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"31'3","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:57:05.325834+0000","last_change":"2024-09-15T21:57:05.325864+0000","last_active":"2024-09-15T21:57:05.325834+0000","last_peered":"2024-09-15T21:57:05.325834+0000","last_clean":"2024-09-15T21:57:05.325834+0000","last_became_active":"2024-09-15T21:56:49.732754+0000","last_became_peered":"2024-09-15T21:56:49.732754+0000","last_unstale":"2024-09-15T21:57:05.325834+0000","last_undegraded":"2024-09-15T21:57:05.325834+0000","last_fullsized":"2024-09-15T21:57:05.325834+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-09-15T21:57:05.325740+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-09-15T21:57:05.325740+0000","last_clean_scrub_stamp":"2024-09-15T21:57:05.325740+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:46:35.128286+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00086675000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:57:03.996595+0000","last_change":"2024-09-15T21:57:03.996620+0000","last_active":"2024-09-15T21:57:03.996595+0000","last_peered":"2024-09-15T21:57:03.996595+0000","last_clean":"2024-09-15T21:57:03.996595+0000","last_became_active":"2024-09-15T21:56:49.733153+0000","last_became_peered":"2024-09-15T21:56:49.733153+0000","last_unstale":"2024-09-15T21:57:03.996595+0000","last_undegraded":"2024-09-15T21:57:03.996595+0000","last_fullsized":"2024-09-15T21:57:03.996595+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:57:03.996502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:57:03.996502+0000","last_clean_scrub_stamp":"2024-09-15T21:57:03.996502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:29.831402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056327000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:57:04.341115+0000","last_change":"2024-09-15T21:57:04.341167+0000","last_active":"2024-09-15T21:57:04.341115+0000","last_peered":"2024-09-15T21:57:04.341115+0000","last_clean":"2024-09-15T21:57:04.341115+0000","last_became_active":"2024-09-15T21:56:49.731371+0000","last_became_peered":"2024-09-15T21:56:49.731371+0000","last_unstale":"2024-09-15T21:57:04.341115+0000","last_undegraded":"2024-09-15T21:57:04.341115+0000","last_fullsized":"2024-09-15T21:57:04.341115+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:57:04.341016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:57:04.341016+0000","last_clean_scrub_stamp":"2024-09-15T21:57:04.341016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:53:20.223290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037233999999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"31'2","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:57:06.370964+0000","last_change":"2024-09-15T21:57:06.371006+0000","last_active":"2024-09-15T21:57:06.370964+0000","last_peered":"2024-09-15T21:57:06.370964+0000","last_clean":"2024-09-15T21:57:06.370964+0000","last_became_active":"2024-09-15T21:56:49.732855+0000","last_became_peered":"2024-09-15T21:56:49.732855+0000","last_unstale":"2024-09-15T21:57:06.370964+0000","last_undegraded":"2024-09-15T21:57:06.370964+0000","last_fullsized":"2024-09-15T21:57:06.370964+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-15T21:57:06.370849+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-15T21:57:06.370849+0000","last_clean_scrub_stamp":"2024-09-15T21:57:06.370849+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:51.727096+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0011226420000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"31'64","reported_seq":156,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:57:04.401670+0000","last_change":"2024-09-15T21:57:04.401721+0000","last_active":"2024-09-15T21:57:04.401670+0000","last_peered":"2024-09-15T21:57:04.401670+0000","last_clean":"2024-09-15T21:57:04.401670+0000","last_became_active":"2024-09-15T21:56:47.710235+0000","last_became_peered":"2024-09-15T21:56:47.710235+0000","last_unstale":"2024-09-15T21:57:04.401670+0000","last_undegraded":"2024-09-15T21:57:04.401670+0000","last_fullsized":"2024-09-15T21:57:04.401670+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'64","last_scrub_stamp":"2024-09-15T21:57:04.401567+0000","last_deep_scrub":"31'64","last_deep_scrub_stamp":"2024-09-15T21:57:04.401567+0000","last_clean_scrub_stamp":"2024-09-15T21:57:04.401567+0000","objects_scrubbed":2,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:23:08.301324+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":27,"seq":115964117001,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":266240,"data_stored":90647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":27,"seq":115964117001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":262144,"data_stored":90628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":25,"seq":107374182409,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":840,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":860160,"data_stored":681015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182409,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":836,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":856064,"data_stored":680996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:57:24.093 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T21:57:24.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.193+0000 7f4019b6c640 1 Processor -- start 2024-09-15T21:57:24.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f4019b6c640 1 -- start start 2024-09-15T21:57:24.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f4019b6c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 0x7f4014095c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:24.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f4019b6c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40140961d0 0x7f4014096650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:24.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f4019b6c640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40141321c0 0x7f40141345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:24.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f4019b6c640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f40140758a0 con 0x7f40141321c0 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f4019b6c640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f4014096b90 con 0x7f40140958b0 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f4019b6c640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f40140739a0 con 0x7f40140961d0 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f4013fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40141321c0 0x7f40141345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f400affd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40140961d0 0x7f4014096650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f40137fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 0x7f4014095c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f4013fff640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f40141321c0 0x7f40141345b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57774/0 (socket says 172.21.15.47:57774) 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f400affd640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40140961d0 0x7f4014096650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3302/0 says I am v2:172.21.15.47:59818/0 (socket says 172.21.15.47:59818) 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f4013fff640 1 -- 172.21.15.47:0/3906725038 learned_addr learned my addr 172.21.15.47:0/3906725038 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f40137fe640 1 -- 172.21.15.47:0/3906725038 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40140961d0 msgr2=0x7f4014096650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f40137fe640 1 --2- 172.21.15.47:0/3906725038 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40140961d0 0x7f4014096650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f40137fe640 1 -- 172.21.15.47:0/3906725038 >> v2:172.21.15.47:3300/0 conn(0x7f40141321c0 msgr2=0x7f40141345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:57:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f40137fe640 1 --2- 172.21.15.47:0/3906725038 >> v2:172.21.15.47:3300/0 conn(0x7f40141321c0 0x7f40141345b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.194+0000 7f40137fe640 1 -- 172.21.15.47:0/3906725038 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4014097330 con 0x7f40140958b0 2024-09-15T21:57:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.195+0000 7f4013fff640 1 --2- 172.21.15.47:0/3906725038 >> v2:172.21.15.47:3300/0 conn(0x7f40141321c0 0x7f40141345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.195+0000 7f400affd640 1 --2- 172.21.15.47:0/3906725038 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40140961d0 0x7f4014096650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.195+0000 7f40137fe640 1 --2- 172.21.15.47:0/3906725038 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 0x7f4014095c90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f40040097a0 tx=0x7f4004029fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2b6adb0e3b330cb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.195+0000 7f4012ffd640 1 -- 172.21.15.47:0/3906725038 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004037070 con 0x7f40140958b0 2024-09-15T21:57:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.195+0000 7f4012ffd640 1 -- 172.21.15.47:0/3906725038 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400402ad20 con 0x7f40140958b0 2024-09-15T21:57:24.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.195+0000 7f4012ffd640 1 -- 172.21.15.47:0/3906725038 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004032660 con 0x7f40140958b0 2024-09-15T21:57:24.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.196+0000 7f4019b6c640 1 -- 172.21.15.47:0/3906725038 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 msgr2=0x7f4014095c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:24.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.196+0000 7f4019b6c640 1 --2- 172.21.15.47:0/3906725038 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 0x7f4014095c90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f40040097a0 tx=0x7f4004029fd0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.196+0000 7f4019b6c640 1 -- 172.21.15.47:0/3906725038 shutdown_connections 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.196+0000 7f4019b6c640 1 --2- 172.21.15.47:0/3906725038 >> v2:172.21.15.47:3300/0 conn(0x7f40141321c0 0x7f40141345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.196+0000 7f4019b6c640 1 --2- 172.21.15.47:0/3906725038 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40140961d0 0x7f4014096650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.196+0000 7f4019b6c640 1 --2- 172.21.15.47:0/3906725038 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 0x7f4014095c90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.196+0000 7f4019b6c640 1 -- 172.21.15.47:0/3906725038 >> 172.21.15.47:0/3906725038 conn(0x7f401409c350 msgr2=0x7f401409c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.196+0000 7f4019b6c640 1 -- 172.21.15.47:0/3906725038 shutdown_connections 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.196+0000 7f4019b6c640 1 -- 172.21.15.47:0/3906725038 wait complete. 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.197+0000 7f4019b6c640 1 Processor -- start 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.197+0000 7f4019b6c640 1 -- start start 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.197+0000 7f4019b6c640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 0x7f4014130860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.197+0000 7f4019b6c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40140961d0 0x7f4014130da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.197+0000 7f4019b6c640 1 --2- >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40141321c0 0x7f40141312e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.197+0000 7f4019b6c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4014134e80 con 0x7f40140961d0 2024-09-15T21:57:24.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.197+0000 7f4019b6c640 1 -- --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_getmap magic: 0 -- 0x7f4014134d00 con 0x7f40140958b0 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.197+0000 7f4019b6c640 1 -- --> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] -- mon_getmap magic: 0 -- 0x7f4014135000 con 0x7f40141321c0 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f400affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40140961d0 0x7f4014130da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f40137fe640 1 --2- >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 0x7f4014130860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f400affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40140961d0 0x7f4014130da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57784/0 (socket says 172.21.15.47:57784) 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f400affd640 1 -- 172.21.15.47:0/3717967649 learned_addr learned my addr 172.21.15.47:0/3717967649 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f4013fff640 1 --2- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40141321c0 0x7f40141312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f40137fe640 1 -- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40141321c0 msgr2=0x7f40141312e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f40137fe640 1 --2- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40141321c0 0x7f40141312e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f40137fe640 1 -- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40140961d0 msgr2=0x7f4014130da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f40137fe640 1 --2- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40140961d0 0x7f4014130da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f40137fe640 1 -- 172.21.15.47:0/3717967649 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40141243e0 con 0x7f40140958b0 2024-09-15T21:57:24.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f400affd640 1 --2- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40140961d0 0x7f4014130da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:57:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f40137fe640 1 --2- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 0x7f4014130860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4004031000 tx=0x7f40040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:57:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f4010ff9640 1 -- 172.21.15.47:0/3717967649 <== mon.1 v2:172.21.15.47:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004041070 con 0x7f40140958b0 2024-09-15T21:57:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f4010ff9640 1 -- 172.21.15.47:0/3717967649 <== mon.1 v2:172.21.15.47:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400403c460 con 0x7f40140958b0 2024-09-15T21:57:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f4019b6c640 1 -- 172.21.15.47:0/3717967649 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4014072f10 con 0x7f40140958b0 2024-09-15T21:57:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f4010ff9640 1 -- 172.21.15.47:0/3717967649 <== mon.1 v2:172.21.15.47:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004037040 con 0x7f40140958b0 2024-09-15T21:57:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.198+0000 7f4019b6c640 1 -- 172.21.15.47:0/3717967649 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f40141239e0 con 0x7f40140958b0 2024-09-15T21:57:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.199+0000 7f4019b6c640 1 -- 172.21.15.47:0/3717967649 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4014095c90 con 0x7f40140958b0 2024-09-15T21:57:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.199+0000 7f4010ff9640 1 -- 172.21.15.47:0/3717967649 <== mon.1 v2:172.21.15.47:3301/0 4 ==== mgrmap(e 10) ==== 55999+0+0 (secure 0 0 0) 0x7f4004044070 con 0x7f40140958b0 2024-09-15T21:57:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.200+0000 7f4010ff9640 1 -- 172.21.15.47:0/3717967649 <== mon.1 v2:172.21.15.47:3301/0 5 ==== osd_map(31..31 src has 1..31) ==== 3027+0+0 (secure 0 0 0) 0x7f400407df80 con 0x7f40140958b0 2024-09-15T21:57:24.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.203+0000 7f4010ff9640 1 -- 172.21.15.47:0/3717967649 <== mon.1 v2:172.21.15.47:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+151260 (secure 0 0 0) 0x7f40141243e0 con 0x7f40140958b0 2024-09-15T21:57:24.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.350+0000 7f4019b6c640 1 -- 172.21.15.47:0/3717967649 --> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f4014068de0 con 0x7f40140958b0 2024-09-15T21:57:24.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.354+0000 7f4010ff9640 1 -- 172.21.15.47:0/3717967649 <== mon.1 v2:172.21.15.47:3301/0 7 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f400402ad20 con 0x7f40140958b0 2024-09-15T21:57:24.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.354+0000 7f4010ff9640 1 -- 172.21.15.47:0/3717967649 <== mon.1 v2:172.21.15.47:3301/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f400407d030 con 0x7f40140958b0 2024-09-15T21:57:24.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.356+0000 7f4019b6c640 1 -- 172.21.15.47:0/3717967649 >> v1:172.21.15.47:6800/1790418198 conn(0x7f3ff4047fd0 legacy=0x7f3ff404a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:24.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.356+0000 7f4019b6c640 1 -- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 msgr2=0x7f4014130860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:57:24.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.356+0000 7f4019b6c640 1 --2- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 0x7f4014130860 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4004031000 tx=0x7f40040046c0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.356+0000 7f4019b6c640 1 -- 172.21.15.47:0/3717967649 shutdown_connections 2024-09-15T21:57:24.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.356+0000 7f4019b6c640 1 --2- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3302/0,v1:172.21.15.47:6791/0] conn(0x7f40141321c0 0x7f40141312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.356+0000 7f4019b6c640 1 --2- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40140961d0 0x7f4014130da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.356+0000 7f4019b6c640 1 --2- 172.21.15.47:0/3717967649 >> [v2:172.21.15.47:3301/0,v1:172.21.15.47:6790/0] conn(0x7f40140958b0 0x7f4014130860 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:57:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.356+0000 7f4019b6c640 1 -- 172.21.15.47:0/3717967649 >> 172.21.15.47:0/3717967649 conn(0x7f401409c350 msgr2=0x7f4014071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:57:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.356+0000 7f4019b6c640 1 -- 172.21.15.47:0/3717967649 shutdown_connections 2024-09-15T21:57:24.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-15T21:57:24.356+0000 7f4019b6c640 1 -- 172.21.15.47:0/3717967649 wait complete. 2024-09-15T21:57:24.369 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T21:57:24.369 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T21:57:24.370 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T21:57:24.370 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:57:24.407 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T21:57:24.408 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T21:57:24.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:57:24.476 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T21:57:24.476 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T21:57:24.476 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:57:24.549 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T21:57:24.549 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T21:57:24.549 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:57:24.617 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T21:57:24.617 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T21:57:24.618 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T21:57:24.618 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:57:24.682 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T21:57:24.683 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-15T21:57:24.683 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:57:24.743 INFO:tasks.ceph.mgr.y:Stopped 2024-09-15T21:57:24.744 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T21:57:24.744 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:57:24.744 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:57:24.796 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:57:24.796 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:57:24.796 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:57:24.846 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:57:24.847 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:57:24.847 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:57:24.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-15T21:57:24.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-15T21:57:24.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-15T21:57:24.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-15T21:57:24.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-15T21:57:24.899 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:57:24.899 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T21:57:24.899 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(FS_' | egrep -v '\(MDS_' | egrep -v '\(OSD_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'Monitor daemon marked osd' | egrep -v 'Behind on trimming' | egrep -v 'Manager daemon' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'scrub mismatch' | egrep -v ScrubResult | egrep -v 'wrongly marked' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_MSGR2_NOT_ENABLED\)' | egrep -v 'slow request' | head -n 1 2024-09-15T21:57:24.982 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:57:24.982 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T21:57:25.165 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:57:25.165 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T21:57:25.255 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:57:25.255 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T21:57:25.322 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:57:25.322 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T21:57:25.372 INFO:tasks.ceph:Archiving mon data... 2024-09-15T21:57:25.373 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906220/data/mon.a.tgz 2024-09-15T21:57:25.373 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-09-15T21:57:25.389 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.Fib1ZS3PGs 2024-09-15T21:57:25.389 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Fib1ZS3PGs 2024-09-15T21:57:25.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.Fib1ZS3PGs 2024-09-15T21:57:25.596 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.Fib1ZS3PGs is 215KB 2024-09-15T21:57:25.689 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.Fib1ZS3PGs 2024-09-15T21:57:25.704 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906220/data/mon.b.tgz 2024-09-15T21:57:25.704 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-09-15T21:57:25.759 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.vzhQvxPg4l 2024-09-15T21:57:25.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.vzhQvxPg4l 2024-09-15T21:57:25.873 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.vzhQvxPg4l 2024-09-15T21:57:25.957 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.vzhQvxPg4l is 249KB 2024-09-15T21:57:26.051 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.vzhQvxPg4l 2024-09-15T21:57:26.066 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906220/data/mon.c.tgz 2024-09-15T21:57:26.066 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-09-15T21:57:26.120 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.plN2uasDIJ 2024-09-15T21:57:26.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.plN2uasDIJ 2024-09-15T21:57:26.236 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.plN2uasDIJ 2024-09-15T21:57:26.320 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.plN2uasDIJ is 249KB 2024-09-15T21:57:26.414 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.plN2uasDIJ 2024-09-15T21:57:26.430 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T21:57:26.430 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:57:26.551 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:57:26.551 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T21:57:26.553 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906220/remote/smithi047/crash 2024-09-15T21:57:26.553 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:57:26.615 INFO:tasks.ceph:Compressing logs... 2024-09-15T21:57:26.616 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:57:26.681 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T21:57:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46456.log 2024-09-15T21:57:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40639.log 2024-09-15T21:57:26.682 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.46456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39196.log 2024-09-15T21:57:26.729 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40639.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46456.log.gz 2024-09-15T21:57:26.729 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41296.log 2024-09-15T21:57:26.729 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39196.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40639.log.gz 2024-09-15T21:57:26.729 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38944.log 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41296.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39196.log.gz 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45509.log 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41296.log.gz 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41385.log 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45509.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38944.log.gz 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39002.log 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45509.log.gz 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45925.log 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41385.log.gz 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39237.log 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45925.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39002.log.gz 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41118.log 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45925.log.gz 2024-09-15T21:57:26.730 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38668.log 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39237.log.gz 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40369.log 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38668.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41118.log.gz 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40369.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38668.log.gz 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32561.log 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40369.log.gz 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41217.log 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45532.log 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41217.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32561.log.gz 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45926.log 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41217.log.gz 2024-09-15T21:57:26.731 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46244.log 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45532.log.gz 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39154.log 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45926.log.gz 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38262.log 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46244.log: /var/log/ceph/ceph-client.admin.39154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46414.log 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46244.log.gz 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39154.log.gz 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45758.log 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46414.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.38262.log.gz 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39004.log 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46414.log.gz 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41511.log 2024-09-15T21:57:26.732 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45758.log: /var/log/ceph/ceph-client.admin.39004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40880.log 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45758.log.gz 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39004.log.gz 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38795.log 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41101.log 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40880.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41511.log.gz 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41766.log 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40880.log.gz 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38795.log.gz 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:gzip/var/log/ceph/ceph-client.admin.41101.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45285.log 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46888.log 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41101.log.gz 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6%/var/log/ceph/ceph-client.admin.45285.log: -- replaced with /var/log/ceph/ceph-client.admin.41766.log.gz 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41237.log 2024-09-15T21:57:26.733 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45285.log.gz 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41665.log 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46888.log.gz 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38398.log 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41237.log: /var/log/ceph/ceph-client.admin.41665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38667.log 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41237.log.gz 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39287.log 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46583.log 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38667.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38398.log.gz 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39287.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38667.log.gz 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39003.log 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39287.log.gz 2024-09-15T21:57:26.734 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40511.log 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46583.log: /var/log/ceph/ceph-client.admin.39003.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26392.log 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46583.log.gz 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39003.log.gz 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46075.log 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46633.log 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40511.log.gz 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.tmp-client.admin.26392.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26392.log.gz 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46075.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46075.log.gz 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47014.log 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46633.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38237.log 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46633.log.gz 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38073.log 2024-09-15T21:57:26.735 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47014.log.gz 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41859.log 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40411.log 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38073.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38237.log.gz 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46930.log 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38073.log.gz 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr: 94.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41859.log.gz 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46972.log 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46930.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40411.log.gz 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32759.log 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.46930.log.gz 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45486.log 2024-09-15T21:57:26.736 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46972.log.gz 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38796.log 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32759.log: /var/log/ceph/ceph-client.admin.45486.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32759.log.gz 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45927.log 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45486.log.gz 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45325.log 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38796.log: /var/log/ceph/ceph-client.admin.45927.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38796.log.gz 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45927.log.gz 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32936.log 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45325.log.gz 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46804.log 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46846.log 2024-09-15T21:57:26.737 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32936.log: /var/log/ceph/ceph-client.admin.46804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45616.log 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32936.log.gz 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46804.log.gz 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46762.log 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38181.log 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46846.log.gz 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45616.log: /var/log/ceph/ceph-client.admin.46762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45445.log 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45616.log.gz 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38181.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46762.log.gz 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38181.log.gz 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32601.log 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41469.log 2024-09-15T21:57:26.738 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mon.a.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45445.log.gz 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46541.log 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32601.log.gz 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:gzip/var/log/ceph/ceph-client.admin.41469.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38500.log 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46541.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41469.log.gz 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40950.log 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46541.log.gz 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38500.log: gzip -5 --verbose 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38500.log.gz 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr: -- /var/log/ceph/ceph-client.admin.38314.log 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32242.log 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40950.log.gz 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42791.log 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38314.log.gz 2024-09-15T21:57:26.739 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41585.log 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42791.log: gzip 92.2% -- replaced with /var/log/ceph/ceph-client.admin.32242.log.gz 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41585.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42791.log.gz 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47096.log 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41585.log.gz 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45574.log 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41427.log 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47096.log.gz 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46264.log 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45574.log.gz 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38499.log 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41427.log.gz 2024-09-15T21:57:26.740 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40907.log 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46264.log.gz 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40642.log 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38499.log.gz 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40514.log 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40907.log.gz 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38356.log 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40642.log.gz 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40514.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.40514.log.gz 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46720.log 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38356.log.gz 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40507.log 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.46720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40512.log 2024-09-15T21:57:26.741 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46720.log.gz/var/log/ceph/ceph-client.admin.40507.log: 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40507.log.gz 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41297.log 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38497.log 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41297.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.40512.log.gz 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41297.log.gz 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38836.log 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38666.log 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38836.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38497.log.gz 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38836.log.gz 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32641.log 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38666.log.gz 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40790.log 2024-09-15T21:57:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32641.log: 93.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32641.log.gz 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40284.log 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41038.log 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40284.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40790.log.gz 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40284.log.gz 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41038.log: 91.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41038.log.gz 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45757.log 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46076.log 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46678.log 2024-09-15T21:57:26.743 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45757.log.gz 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45365.log 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46678.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46076.log.gz 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46678.log.gz 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46096.log 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45365.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45365.log.gz 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45019.log 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39112.log 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46096.log.gz 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46372.log 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39112.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45019.log.gz 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39112.log.gz 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46372.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46372.log.gz 2024-09-15T21:57:26.744 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45658.log 2024-09-15T21:57:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45928.log 2024-09-15T21:57:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.45658.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45658.log.gz 2024-09-15T21:57:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45405.log 2024-09-15T21:57:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45928.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45928.log.gz 2024-09-15T21:57:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46243.log 2024-09-15T21:57:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45405.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45405.log.gz 2024-09-15T21:57:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38498.log 2024-09-15T21:57:26.745 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46243.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46243.log.gz 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38835.log 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38498.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38498.log.gz 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45759.log 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38835.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38835.log.gz 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40682.log 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45759.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.45759.log.gz 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40681.log 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40682.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40682.log.gz 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46498.log 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40681.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40681.log.gz 2024-09-15T21:57:26.756 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46498.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46498.log.gz 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45760.log 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.3.log: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46074.log 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.45760.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45760.log.gz 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41094.log 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46074.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46074.log.gz 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47056.log 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41094.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41094.log.gz 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38665.log 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.47056.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47056.log.gz 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32521.log 2024-09-15T21:57:26.757 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38665.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38665.log.gz 2024-09-15T21:57:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46242.log 2024-09-15T21:57:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32521.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32521.log.gz 2024-09-15T21:57:26.758 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.46242.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46242.log.gz 2024-09-15T21:57:26.758 INFO:teuthology.orchestra.run.smithi047.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T21:57:26.760 INFO:teuthology.orchestra.run.smithi047.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T21:57:26.772 INFO:teuthology.orchestra.run.smithi047.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T21:57:26.778 INFO:teuthology.orchestra.run.smithi047.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T21:57:26.792 INFO:teuthology.orchestra.run.smithi047.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T21:57:26.803 INFO:teuthology.orchestra.run.smithi047.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T21:57:26.828 INFO:teuthology.orchestra.run.smithi047.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T21:57:26.829 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-09-15T21:57:26.830 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m0.158s 2024-09-15T21:57:26.830 INFO:teuthology.orchestra.run.smithi047.stderr:user 0m0.620s 2024-09-15T21:57:26.830 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m0.107s 2024-09-15T21:57:26.830 INFO:tasks.ceph:Archiving logs... 2024-09-15T21:57:26.830 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906220/remote/smithi047/log 2024-09-15T21:57:26.831 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:57:27.068 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T21:57:27.078 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T21:57:27.079 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:57:27.131 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T21:57:27.141 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T21:57:27.141 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:57:27.160 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:57:27.165 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:57:27.165 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-09-15T21:57:27.165 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 +1386us[+1403us] +/- 57ms 2024-09-15T21:57:27.165 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 -403us[ -386us] +/- 52ms 2024-09-15T21:57:27.166 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 -1033us[-1017us] +/- 62ms 2024-09-15T21:57:27.166 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:57:27.166 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T21:57:27.175 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T21:57:27.176 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T21:57:27.185 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T21:57:27.271 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:57:27.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:57:27.299 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T21:57:27.310 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T21:57:27.320 INFO:teuthology.task.internal:Duration was 505.466694 seconds 2024-09-15T21:57:27.320 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T21:57:27.339 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T21:57:27.339 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:57:27.379 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:57:27.851 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T21:57:27.851 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-09-15T21:57:27.852 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:57:27.914 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T21:57:27.914 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:57:27.983 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T21:57:27.983 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:57:28.130 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T21:57:28.141 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T21:57:28.141 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:57:28.167 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T21:57:28.178 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:57:28.231 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-09-15T21:57:28.263 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:57:28.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:57:28.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T21:57:28.330 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T21:57:28.331 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906220/remote/smithi047 2024-09-15T21:57:28.331 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:57:28.394 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T21:57:28.394 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:57:28.444 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T21:57:28.456 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T21:57:28.456 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T21:57:28.466 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T21:57:28.466 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:57:28.499 INFO:teuthology.orchestra.run.smithi047.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:57 /home/ubuntu/cephtest 2024-09-15T21:57:28.500 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T21:57:28.511 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T21:57:28.535 INFO:teuthology.run:Summary data: description: fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} duration: 505.466694355011 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T21:57:28.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:57:28.616 INFO:teuthology.run:pass